system

Abstract

We provide the system. [Solution] A means for the user to upload room plans and photos, A server receives and analyzes uploaded drawings and photographs to recognize the room layout, A means by which the user inputs their preferred taste and budget, and a means by which the server receives that information. A server provides a means for generating multiple coordination proposals using an AI model based on input information and floor plans, The server provides a means to add information on the source and price of the furniture used in the coordination plan, The device provides a means for displaying outfit suggestions, purchase locations, price information, and online purchase links. A means by which a user selects a displayed coordination plan and purchases furniture, and a means of updating that plan on the server. A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In room decoration or furniture selection for a new home, there is a problem that many people are troubled by the placement and coordination methods. There are also problems such as not knowing where furniture that matches one's own taste is sold, or not being able to find furniture that meets the budget. For busy modern people, these problems become factors that further increase the burden.

Means for Solving the Problems

[0005] The present invention solves the above problems by providing a system that includes means for a user to upload a floor plan or photograph of a room, means for a server to receive and analyze the uploaded floor plan or photograph to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using an AI model based on the input information and the floor plan, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposal, means for a terminal to display the coordination proposal, where to purchase, price information, and online purchase links, and means for the user to select a displayed coordination proposal, purchase the furniture, and update the server accordingly.

[0006] A "user" refers to an individual or corporation that uses the system to request room coordination services.

[0007] A "drawing" refers to a file containing two-dimensional or three-dimensional visual information that shows the layout of a room or the arrangement of furniture.

[0008] "Photo" refers to an image file that captures the current state of the room.

[0009] A "server" refers to a central processing unit that receives and analyzes drawings and photographs, and generates and transmits coordination proposals.

[0010] "Analysis" refers to the process by which the server automatically recognizes the room layout and other characteristics from uploaded drawings and photographs.

[0011] "Floor plan" refers to a layout that includes detailed information such as the structure and arrangement of rooms, walls, windows, doors, and furniture placement.

[0012] "Preferred style" refers to the style and atmosphere of a room that the user desires, such as a specific design theme like "resort hotel style."

[0013] "Budget" refers to the maximum amount of money a user can spend on decorating their room.

[0014] "Input" refers to the act of a user providing information such as their preferred taste and budget to the system via their device.

[0015] An "AI model" refers to an artificial intelligence algorithm that learns from a large amount of data and automatically generates room coordination suggestions.

[0016] A "coordination plan" refers to a furniture arrangement and interior design plan proposed based on the analyzed floor plan and the user's preferences.

[0017] "Purchase source" refers to online shops or physical stores where you can buy the suggested furniture and interior items.

[0018] "Pricing information" refers to data showing the purchase cost of the suggested furniture and interior items.

[0019] An "online purchase link" refers to a URL that provides access to an internet site where you can directly purchase the suggested furniture and interior items online.

[0020] "Terminal" refers to electronic devices such as computers, smartphones, and tablets that users operate.

[0021] "Selection" refers to the act of a user deciding on a specific outfit from among the outfit suggestions provided by the system.

[0022] "Updating" refers to the act of continuously sending and recording information such as user selections and purchase history to the server. [Brief explanation of the drawing]

[0023] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0024] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0025] First, let's explain the terminology used in the following explanation.

[0026] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0027] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0028] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0029] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0031] [First Embodiment]

[0032] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0033] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0036] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0039] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0043] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0044] This invention is a system in which users upload floor plans or photos of their rooms, and an AI model is used to generate room coordination proposals based on these. The system proposes multiple coordination proposals that match the user's preferred style and budget, and also provides information on where to purchase the suggested furniture and their prices. The system's program processing is described below in natural language.

[0045] 1. The user uploads a floor plan or photos of the room.

[0046] Users upload room layouts and photos to the system using their devices. The uploaded layouts and photos are then sent from the device to the server.

[0047] 2. The server analyzes the drawings and photographs.

[0048] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes the room's shape, the location of walls, windows, and doors. This analysis uses image recognition algorithms to extract the accurate layout.

[0049] 3. The user enters their preferred taste and budget.

[0050] The user accesses a form on their device to enter their preferred style (e.g., "resort hotel style") and budget. The information entered by the user is sent to the server via the device.

[0051] 4. The server generates a coordination proposal.

[0052] The server uses an AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model suggests optimal furniture placement and interior design based on past data and trends.

[0053] 5. The server adds information about the supplier and price.

[0054] The server adds information about the purchase locations and prices of the furniture used in the generated coordination proposals. This involves retrieving specific store information and online purchase links from the database and integrating them into the proposal.

[0055] 6. The device displays outfit suggestions.

[0056] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[0057] 7. The user selects and purchases their favorite outfit.

[0058] Users select their preferred outfit from several displayed coordination options. They can easily purchase the selected furniture by clicking the displayed online purchase link. The user's selection history and purchase information are sent from their device to the server and used to optimize future suggestions.

[0059] Specific example

[0060] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. The user can then easily redecorate their room by selecting their favorite design and purchasing the furniture via the displayed link.

[0061] This system allows users to easily coordinate their rooms to their liking and receive support in selecting the best furniture within their budget. The server's AI model constantly learns the latest trends and data, ensuring that the suggested coordination ideas are of high quality and relevant to the times.

[0062] The following describes the processing flow.

[0063] Step 1:

[0064] The user selects room floor plans and photo files using their device and clicks the upload button.

[0065] Step 2:

[0066] The terminal sends the selected drawing or photo file to the server.

[0067] Step 3:

[0068] The server receives drawing and photo files sent from the terminal.

[0069] Step 4:

[0070] The server uses image analysis algorithms to recognize the room layout from the received drawings or photographs. This includes the process of extracting the locations of walls, windows, doors, and furniture.

[0071] Step 5:

[0072] The user accesses a form on their device interface where they can enter their preferred style (e.g., "resort hotel style") and budget.

[0073] Step 6:

[0074] The user enters the required information and clicks the submit button.

[0075] Step 7:

[0076] The device sends user-entered preferences and budget information to the server.

[0077] Step 8:

[0078] The server analyzes the customer's preferred taste and budget information that it receives.

[0079] Step 9:

[0080] Based on floor plan information recognized by the server, as well as the user's preferred style and budget, an AI model is used to generate multiple design options.

[0081] Step 10:

[0082] For each coordinated outfit design generated by the server, the server retrieves information from the database regarding the source and price of the furniture used.

[0083] Step 11:

[0084] The server retrieves the supplier and price information and adds it to each suggested outfit. If online purchase is possible, a link to that purchase is also included.

[0085] Step 12:

[0086] The server sends the outfit suggestions, purchase locations, pricing information, and online purchase links to the device.

[0087] Step 13:

[0088] The terminal receives the suggested outfit from the server and displays it to the user in a visually easy-to-understand format.

[0089] Step 14:

[0090] The user selects their favorite outfit from several displayed outfit options.

[0091] Step 15:

[0092] The user clicks the online purchase link for their chosen interior design and buys the furniture online.

[0093] Step 16:

[0094] The device sends the user's selections and purchase history to the server, which then updates that information.

[0095] (Example 1)

[0096] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] Traditionally, it has been time-consuming and laborious for users to coordinate a room to their liking. Furthermore, a lack of knowledge and skills in selecting and arranging appropriate furniture made achieving an ideal coordination difficult. Additionally, choosing the best furniture within a budget was challenging, often limiting options. As a result, users have struggled to achieve a satisfactory room design.

[0098] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0099] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using a generation AI model based on the input information and the layout, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, means for the user to select a displayed coordination proposal and purchase the furniture and update the server accordingly, means for the server to generate prompt messages based on the layout information analyzed by the server and the user's input information and send them to the generation AI model, and means for the server to generate the optimal coordination proposal based on trained data and trend data in the generation AI model. This makes it possible for users to easily create a room coordination that suits their preferences within their budget.

[0100] "A means for users to upload room plans and photos" refers to an interface that allows users to select room plans and photos from their devices and send them to the server.

[0101] "A method for a server to receive and analyze uploaded drawings and photos to recognize the layout of a room" refers to a technology in which a server receives drawings and photos sent by a user and uses an image recognition algorithm to extract layout information such as the shape of the room and the location of walls, windows, and doors.

[0102] "A means for users to input their preferred taste and budget, and a means for the server to receive that information" refers to a system in which users input their preferred interior style and budget through a form displayed on their device, and the server receives the entered information.

[0103] "A means by which a server generates multiple coordination proposals using a generative AI model based on input information and floor plan" refers to a technology in which a server generates multiple interior coordination proposals using a generative AI model that has learned from past data and the latest trends, based on user input information and analyzed floor plan information.

[0104] "A means by which the server adds information on the source and price of furniture used in a coordination plan" refers to a technology in which the server retrieves information on the source and price of furniture used in a generated coordination plan from a database and integrates it into the coordination plan.

[0105] "Means by which a terminal displays coordination suggestions, purchase locations, price information, and online purchase links" refers to an interface that allows the terminal to visually display coordination suggestions, furniture purchase locations, price information, and online purchase links received from the server to the user.

[0106] "A means for a user to select a displayed coordination plan and purchase furniture, and a means for updating that information on a server" refers to a system for a user to select a coordination plan they like from the displayed options and purchase furniture through an online purchase link, and the technology for sending and updating purchase information on a server.

[0107] "A means of generating prompt text based on floor plan information analyzed by the server and user input information, and sending it to a generation AI model" refers to a technology that generates prompt text to be input to a generation AI model based on floor plan information analyzed by the server and user input information, and then inputs that prompt text into the generation AI model.

[0108] "A means by which a server generates optimal coordination proposals based on trained data and trend data from a generated AI model" refers to a technology in which a server uses a generated AI model to consider trained data and the latest trends to generate optimal interior coordination proposals.

[0109] This invention is a system in which a user uploads floor plans or photos of a room, and an AI model is used to generate room coordination suggestions based on that information. The specific steps for implementing this system are shown below.

[0110] Users first upload room layouts and photos using their own devices (computers, smartphones, etc.). This upload is done via an interface using HTML or React. The uploaded layouts and photos are sent from the device to the server, which then receives them.

[0111] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes room shape, wall and window locations, and door positions. Image recognition libraries such as OpenCV and TENSORFLOW® are used for this analysis. The server utilizes image recognition algorithms to extract accurate layouts.

[0112] The user then enters their preferred style (e.g., "resort hotel style") and budget on the on-device interface. A form for this purpose is built using HTML or React, and the information entered by the user is sent to the server via the device.

[0113] The server uses a generative AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. This AI model suggests optimal furniture placement and interior design based on historical and trend data. During the generation process, the server generates prompt messages based on the analyzed layout information and user input, and sends these prompt messages to the generative AI model.

[0114] As a concrete example, if a user requests a "resort hotel style" room design and sets a budget of 50,000 yen, the following prompt message will be generated: "I would like a resort hotel style room design. My budget is 50,000 yen. I have uploaded photos of the room. Based on this information, please generate three optimal design options."

[0115] The server generates optimal coordination suggestions based on data trained by a generation AI model and the latest trend data. Each generated coordination suggestion includes information on where to purchase the furniture used and its price. This allows users to easily create a room coordination that suits their preferences within their budget.

[0116] Ultimately, the device receives the coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links. The user can easily redecorate their room by selecting their favorite from the multiple coordination suggestions displayed and purchasing the furniture through the displayed links.

[0117] As described above, this system provides an effective means for users to easily coordinate their rooms and choose the optimal furniture within their budget.

[0118] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0119] Step 1:

[0120] Users upload room layouts and photos. Specifically, users use their devices to click the "Upload" button and select room layout and photo files. Then, they press the "Submit" button, and the selected files are sent from their device to the server. The input is the room layout and photos provided by the user, and the output is the image files sent to the server.

[0121] Step 2:

[0122] The server analyzes the received drawings and photographs. At this stage, the received image files are analyzed using image recognition libraries such as OpenCV or TensorFlow. The input is the image file sent to the server, and the output is the analyzed room layout information (room shape, location of walls, windows, doors, etc.). The server triggers the "start image recognition" process, analyzing the image pixel by pixel to extract room features.

[0123] Step 3:

[0124] The user inputs their preferred style and budget. Specifically, the user enters a style such as "resort hotel style" and a budget such as "50,000 yen" into a form provided on the device's interface. The form used for this is built with HTML or React. The input is the user's preferred style and budget, and the output is the style and budget information sent from the device to the server.

[0125] Step 4:

[0126] The server uses a generating AI model to create multiple coordination proposals based on the user's preferred taste, budget information, and analyzed floor plan information. The input is the user's taste information, budget information, and floor plan information, and the output is multiple coordination proposals generated by the generating AI model. The server combines the analysis results and the user's wishes to generate a prompt message and input it into the AI ​​model. For example, "I would like a resort hotel-style room coordination. My budget is 50,000 yen. I have uploaded photos of the room. Based on this information, please generate three optimal coordination proposals."

[0127] Step 5:

[0128] The server adds information about the source and price of the furniture used in the generated coordination proposals. The server retrieves source and price information from the database and integrates it into each coordination proposal. The input is the generated coordination proposal, and the output is the coordination proposal with added source and price information. Here, the server queries for specific store information and online shop links and integrates them into the proposal content.

[0129] Step 6:

[0130] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The input is the coordination suggestion information sent from the server, and the output is a detailed furniture layout diagram, purchase location, price information, and online purchase links displayed to the user. This information is displayed to the user on the device using HTML or React.

[0131] Step 7:

[0132] The user selects a displayed coordination plan and purchases the furniture. Specifically, the user selects a preferred coordination plan on their device and clicks the displayed online purchase link to complete the purchase process. The input is the user's selection of coordination plans and purchase process, and the output is the selection history and purchase information sent from the device to the server. The server stores this information in a database and uses it to optimize future suggestions.

[0133] (Application Example 1)

[0134] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0135] In modern life, interior design is an important element for many people, but there is a lack of easy and efficient ways to do it. Furthermore, the process of users selecting interior items that suit their taste and purchasing them within their budget is complex and cumbersome. Moreover, there is currently no system utilizing appropriate AI models to provide design suggestions, and these problems need to be addressed.

[0136] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0137] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using an AI model based on the input information and the layout, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, means for the user to select a displayed coordination proposal and purchase the furniture and update the server accordingly, means for visually displaying the interior design proposals generated by the AI ​​model via a smartphone app to support the purchase, means for displaying suggested content based on the user's preferences, and means for checking the where to purchase and the price of the furniture included in the interior design proposals online and performing the purchase procedure directly via the link. This makes it possible for users to easily check, select, and purchase interior coordination that suits their preferences and budget.

[0138] A "user" is an individual or corporation that uses this system to create room interior design plans or purchase furniture.

[0139] "Room floor plans and photos" refer to data containing visual information that users upload to the system to identify the layout and characteristics of a room.

[0140] A "server" is a central processing unit that analyzes drawings and photos received from users, generates coordination proposals, and also provides information on furniture suppliers and pricing.

[0141] "Analysis" refers to the process by which a server recognizes the layout and interior features of a room based on drawings and photographs.

[0142] "Preferred style and budget" refers to the interior style and amount of money that users can spend, which serve as the basis for generating room coordination suggestions.

[0143] An "AI model" refers to an artificial intelligence algorithm or computational model that uses input information and floor plan information to determine the optimal interior layout and furniture selection.

[0144] A "coordination plan" is a proposal generated by an AI model that shows the arrangement and selection of interior furnishings for a specific room.

[0145] "Purchase location and price information" refers to information about where the furniture included in the coordination plan can be purchased and how much it costs.

[0146] An "online purchase link" is a direct reference URL to a website or online store where users can immediately purchase the suggested furniture.

[0147] A "smartphone app" is application software that users can install to check room interior design ideas or purchase furniture.

[0148] "Selection history" refers to the record data of the coordination ideas and furniture that the user has selected and purchased so far.

[0149] "Optimization" refers to the process of adjusting the next suggestion to better match the user's needs, based on their preferences and past selection history.

[0150] This invention is a system in which a user uploads floor plans or photos of a room, and an AI model is used to generate room coordination suggestions based on that information. Specifically, the invention can be implemented through the following steps.

[0151] Hardware and software to be used

[0152] This system is implemented using the following hardware and software.

[0153] Smartphone: A device for applications that the user installs.

[0154] Server: Central processing unit that performs image analysis and inference processing using AI models.

[0155] OpenCV: A library for servers to perform image processing.

[0156] TensorFlow: A machine learning library for performing inference processing on AI models.

[0157] Database module: Manages furniture purchase information and pricing.

[0158] Program processing

[0159] The processes performed at each step of this system are as follows:

[0160] 1. Users upload room floor plans or photos:

[0161] Users upload room layouts and photos using a smartphone app. The uploaded layouts and photos are then sent from the smartphone to the server.

[0162] 2. The server analyzes the drawings and photos:

[0163] The server uses the OpenCV library to analyze the received drawings and photographs. The analysis automatically recognizes the room layout, extracting information such as the room's shape and the locations of walls, windows, and doors.

[0164] 3. The user enters their preferred taste and budget:

[0165] Users enter their preferred style (e.g., "resort hotel style") and budget on a smartphone app. The information entered by the user is sent to the server via the smartphone.

[0166] 4. The server generates a coordination proposal:

[0167] The server uses an AI model powered by TensorFlow to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model then suggests the optimal furniture placement and interior design based on past data and trends.

[0168] 5. The server adds the supplier and price information:

[0169] The server retrieves and adds information about the purchase locations and prices of the furniture used in the generated coordination plan from a database module. The process then integrates specific store information and online purchase links into the coordination plan.

[0170] 6. Your smartphone will display outfit suggestions:

[0171] The smartphone app receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[0172] 7. The user selects and purchases their favorite outfit:

[0173] Users select their favorite from several displayed coordination options and purchase the furniture via a link displayed through the smartphone app. The purchase information of the selected furniture is updated on the server and used to optimize future suggestions.

[0174] Specific example

[0175] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by information on where to buy it, its price, and an online purchase link. The user can then select their favorite room design and easily redecorate their room by purchasing the furniture via the link displayed on the smartphone app.

[0176] Example of a prompt

[0177] The following input prompts can be set for the generative AI model.

[0178] Prompt: Please provide images and floor plan information for the room. Then, generate the best interior design proposal to match the specified budget and style.

[0179] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0180] Step 1:

[0181] Users upload floor plans and photos of their rooms. Users use their smartphones to take photos and upload them to the server via the application. The input here is the floor plan and photo data sent from the user's device, while the output is the raw image data received on the server side.

[0182] Step 2:

[0183] The server analyzes drawings and photographs to recognize the room layout. The server uses OpenCV to extract structural information about the room within the image and automatically identifies the layout. The input here is the image data received in step 1, and the output is layout information such as the shape of the room and the positions of walls, windows, and doors. Specifically, the server resizes the image and extracts feature points to recognize the room layout.

[0184] Step 3:

[0185] The user enters their preferred taste and budget. The user sets their taste (e.g., "modern" or "classic") and budget within the application and sends this information to the server. The input here is the taste and budget information entered by the user, while the output is the preference and budget data received on the server side.

[0186] Step 4:

[0187] The server generates multiple coordination options using an AI model. Based on the received taste and budget information, and the analyzed floor plan information, the server creates coordination options from the AI ​​model using TensorFlow. The input here is the output data from steps 2 and 3, and the output is the multiple coordination options that have been generated. Specifically, the input data is passed to the AI ​​model, which calculates the optimal furniture placement and interior design.

[0188] Step 5:

[0189] The server adds information about the source and price of the furniture used in the coordination proposal. The server queries the database module for source and price information for the generated coordination proposal and adds the source information and price to each coordination. The input here is the coordination proposal generated in step 4, and the output is data with source and price information for the furniture associated with each coordination added.

[0190] Step 6:

[0191] The device displays coordination suggestions, purchase locations, price information, and online purchase links. The smartphone app visually displays coordination suggestions based on the information received from the server. The input here is the data generated in step 5, and the output is a display of coordination suggestions that the user can view. Specifically, the app screen displays a furniture layout diagram and purchase links for each item.

[0192] Step 7:

[0193] The user selects a displayed coordination suggestion and purchases the furniture. The user chooses their favorite from several displayed coordination suggestions and clicks the online purchase link via the smartphone app to purchase the furniture. The input here is the coordination suggestion displayed in step 6, and the output is information about the purchased items added to the shopping cart. Specifically, the user clicks the displayed link and completes the purchase procedure and payment. The purchase information of the selected furniture is updated on the server and used to optimize future suggestions.

[0194] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0195] This invention is a system in which a user uploads floor plans or photos of a room, and based on that, an AI model and emotion engine are used to generate room coordination suggestions. The system proposes multiple coordination suggestions that match the user's preferred style and budget, and also provides information on where to purchase the suggested furniture and their prices. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, the system optimizes the coordination suggestions according to the user's mood and past emotional data. The system's program processing is described below in natural language.

[0196] 1. The user uploads a floor plan or photos of the room.

[0197] Users upload room layouts and photos to the system using their devices. The uploaded layouts and photos are then sent from the device to the server.

[0198] 2. The server analyzes the drawings and photographs.

[0199] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes the room's shape, the location of walls, windows, and doors. This analysis uses image recognition algorithms to extract the accurate layout.

[0200] 3. The user enters their preferred taste and budget.

[0201] The user accesses a form on their device to enter their preferred style (e.g., "resort hotel style") and budget. The information entered by the user is sent to the server via the device.

[0202] 4. The server generates a coordination proposal.

[0203] The server uses an AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model suggests optimal furniture placement and interior design based on past data and trends.

[0204] 5. The emotion engine recognizes the user's emotions.

[0205] The emotion engine analyzes the user's facial expressions and voice tone through the camera and microphone built into the device to recognize their current emotional state. This allows the server to provide real-time suggestions tailored to the user's mood.

[0206] 6. The server adds information about the supplier and price.

[0207] The server retrieves information on the source and price of the furniture used in the generated coordination plan from its database. This includes retrieving and integrating specific store information and online purchase links.

[0208] 7. The device displays outfit suggestions.

[0209] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[0210] 8. The user selects and purchases their favorite outfit.

[0211] Users select their preferred outfit from several displayed options. In addition, an emotion engine evaluates the user's emotional response to their selection and adjusts the suggestions as needed. Selected furniture can be easily purchased by clicking the displayed online purchase link. The user's selection history and purchase information are sent from the device to the server and used to optimize future suggestions.

[0212] Specific example

[0213] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. Then, an emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, prioritizing and displaying the room design option that best suits their current mood. The user can easily redecorate their room by selecting a specific room design option and purchasing the furniture through the displayed link.

[0214] This system allows users to easily decorate their rooms to their liking, and through real-time suggestion optimization powered by an emotion engine, they can receive optimal suggestions that match their mood. Furthermore, the server's AI model constantly learns the latest trends and data, ensuring that the suggested decorating ideas are of high quality and relevant to the times.

[0215] The following describes the processing flow.

[0216] Step 1:

[0217] The user uses their device to select room floor plans or photo files and clicks the upload button.

[0218] Step 2:

[0219] The terminal sends the selected drawing or photo file to the server.

[0220] Step 3:

[0221] The server receives drawing and photo files sent from the terminal.

[0222] Step 4:

[0223] The server uses image analysis algorithms to recognize the room layout from the received drawings or photographs. This includes the process of extracting the locations of walls, windows, doors, and furniture.

[0224] Step 5:

[0225] The user accesses a form on their device interface to enter their preferred style (e.g., "resort hotel style") and budget.

[0226] Step 6:

[0227] The user enters the required information and clicks the submit button.

[0228] Step 7:

[0229] The device sends user-entered preferences and budget information to the server.

[0230] Step 8:

[0231] The server receives information about the user's preferred tastes and budget.

[0232] Step 9:

[0233] Based on floor plan information recognized by the server, as well as the user's preferred style and budget, an AI model is used to generate multiple design options.

[0234] Step 10:

[0235] For each generated design proposal, the server retrieves information on where to purchase the furniture and its price from the database. This includes obtaining specific store information and online purchase links.

[0236] Step 11:

[0237] The server retrieves the supplier and price information and adds it to each suggested outfit. If online purchase is possible, a link to that purchase is also included.

[0238] Step 12:

[0239] The server sends the outfit suggestions, purchase locations, pricing information, and online purchase links to the device.

[0240] Step 13:

[0241] The terminal receives coordination proposal information sent from the server and displays it to the user in a visually easy-to-understand format.

[0242] Step 14:

[0243] The user selects their favorite outfit from the displayed outfit suggestions.

[0244] Step 15:

[0245] The emotion engine analyzes the user's facial expressions and voice tone through the device's camera and microphone to recognize their current emotional state.

[0246] Step 16:

[0247] The server prioritizes displaying outfit suggestions that best suit the user's current mood, based on their emotional state.

[0248] Step 17:

[0249] The user clicks the online purchase link for the coordination plan they ultimately selected and buys the furniture online.

[0250] Step 18:

[0251] The device sends the user's selections and purchase history to a server, which is then recorded to optimize future suggestions.

[0252] (Example 2)

[0253] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0254] Conventional room coordination suggestion systems failed to consider the user's emotions and moods, making it difficult to increase user satisfaction. Furthermore, information on furniture suppliers and pricing based on the suggested coordination plans was scarce, making it difficult for users to make consistent purchase decisions without hassle. Additionally, there was a challenge in generating and presenting coordination plans that matched the user's preferences and budget in real time.

[0255] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0256] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using an AI model based on the input information and the layout, means for an emotion engine to recognize the user's emotions and optimize the proposals based on that information, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, and means for the user to select a displayed coordination proposal, purchase the furniture, and update the server accordingly. This makes it possible to provide more personalized coordination proposals that take into account the user's emotions and mood, to consistently provide information on where to purchase and the price of the furniture, and to improve user satisfaction.

[0257] - A "user" is the entity that uses this system to coordinate a room.

[0258] A "device" refers to an electronic device used by a user, such as a smartphone or personal computer.

[0259] A "server" is the central processing unit of a system, a computer that receives, analyzes, generates, and transmits data.

[0260] A "drawing" refers to a design plan or floor plan that shows the layout of a room.

[0261] A "photograph" is a still image that captures the current state of the room.

[0262] "Analysis" refers to the process of extracting floor plans and room features from uploaded drawings and photographs.

[0263] "Preferred taste" refers to the type of interior style or design that the user desires.

[0264] "Budget" refers to the maximum amount of money a user is willing to spend on coordinating an outfit.

[0265] An "AI model" is artificial intelligence that learns from past data and trends and generates outfit suggestions based on user input.

[0266] An "emotion engine" is software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[0267] "Purchase location" refers to the stores or online shops where the suggested furniture can be purchased.

[0268] "Price information" refers to information regarding the price of the proposed furniture.

[0269] An "online purchase link" is a URL that allows users to purchase furniture via the internet.

[0270] A "coordination plan" refers to furniture arrangements and interior designs proposed based on the user's preferred style, budget, and room layout.

[0271] "Selection history" refers to information about the coordination ideas and furniture that the user has previously selected in the system.

[0272] "Suggestion optimization" is the process of improving suggestions based on the user's selection history and emotional state.

[0273] This invention relates to a system in which a user uploads floor plans or photos of a room, and based on that, an AI model and an emotion engine are used to generate room coordination suggestions. The system allows the user to input their preferred style and budget, and then proposes multiple coordination suggestions that fit the budget, also providing information on where to purchase the suggested furniture and their prices. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, the system optimizes the coordination suggestions according to the user's mood and past emotional data.

[0274] Hardware and software

[0275] This system is implemented using the following hardware and software:

[0276] Server: This is a central processing unit that receives, analyzes, generates, and transmits data. Specifically, it uses cloud computing services such as Amazon Web Services (AWS®) and Microsoft Azure®.

[0277] Device: A smartphone or computer used by the user. These devices provide the user interface and allow for data uploading and display.

[0278] AI Model: Use generative AI models such as Generative Adversarial Networks (GANs) to generate coordination plans.

[0279] Emotion Engine: Utilize Amazon Rekognition or Microsoft Azure's Emotion API to analyze the user's facial expressions and vocal tones and recognize the emotional state.

[0280] Image Recognition Algorithm: Use OpenCV or TensorFlow to analyze the floor plan of the room from the uploaded drawings or photos.

[0281] Data Processing and Data Calculation

[0282] 1. Data Upload and Reception:

[0283] The user uploads drawings or photos of the room from the terminal. This data is sent to the server via the Internet.

[0284] 2. Image Analysis:

[0285] The server saves the received drawings or photos and uses the image recognition algorithm to analyze the floor plan of the room. As a result, the arrangement of walls, doors, and windows is automatically recognized.

[0286] 3. User Information Input:

[0287] The user inputs their preferred taste and budget into the form on the terminal. This information is sent from the terminal to the server.

[0288] 4. Generation of Coordination Plans:

[0289] The server uses the AI model to generate multiple coordination plans based on the user's input information and the analyzed floor plan data.

[0290] 5. Emotion Recognition:

[0291] The emotion engine analyzes the user's facial expressions and voice tone through the device's camera and microphone, recognizing their emotional state in real time. This information is sent to a server and used to optimize suggestions.

[0292] 6. Add purchase information:

[0293] The server retrieves furniture supplier and price information from the database for each generated coordination plan and integrates it into the plan.

[0294] 7. Display of outfit suggestions:

[0295] The terminal receives coordination proposal information sent from the server and displays it to the user in a visually easy-to-understand format. The display includes furniture layout diagrams, purchase locations, price information, and online purchase links.

[0296] 8. Selection and Purchase:

[0297] Users can purchase furniture online based on their selected coordination plan. Purchase information is sent to the server and stored to optimize future suggestions.

[0298] Specific example

[0299] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. Then, an emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, prioritizing and displaying the room design option that best suits their current mood. The user can easily redecorate their room by selecting a specific room design option and purchasing the furniture through the displayed link.

[0300] Example of a prompt

[0301] Based on the following room photos and floor plan information, please propose three resort-hotel-style living room design ideas within a budget of 50,000 yen. [Attach room photos and floor plan information]

[0302] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0303] Step 1:

[0304] The user uploads room plans and photos using their device. Specifically, the user opens a file selection dialog on their device and selects the room plans and photos. The input here is the room plans and photos, which are sent from the device to the server as an HTTP POST request. The output is the data of the plans and photos transferred to the server.

[0305] Step 2:

[0306] The server analyzes the received drawings and photos. Specifically, the server temporarily stores the received files and uses image recognition algorithms (e.g., OpenCV or TensorFlow) to extract the floor plan information of the room. The input is the uploaded file of the drawing or photo, and the output is the floor plan information (such as the positions of walls, doors, and windows).

[0307] Step 3:

[0308] The user inputs their preferred taste and budget amount through the interface of the terminal. The user enters the taste (e.g., "resort hotel style") and budget into the form and presses the "Send" button. The input is the taste and budget information, which are sent from the terminal to the server. The output is the sent taste and budget data.

[0309] Step 4:

[0310] Based on the received preferred taste and budget information and the analyzed floor plan data, the server uses an AI model to generate multiple coordinate plans. Specifically, a generative AI model such as Generative Adversarial Networks (GANs) is used. The input is the taste, budget information, and floor plan data, and the output is multiple coordinate plans.

[0311] Step 5:

[0312] Through the camera and microphone installed on the terminal, the emotion engine analyzes the user's facial expressions and voice tones and recognizes the emotional state in real time. The input is the data of the user's facial expressions and voice tones, which the emotion engine analyzes. The output is the data of the current emotional state.

[0313] Step 6:

[0314] The server receives emotional state data from the emotion engine and optimizes the suggested outfit combinations. Specifically, this data is used to provide suggestions that are tailored to the user's emotional state in real time. The input is emotional state data, and the output is an outfit combination optimized according to that emotion.

[0315] Step 7:

[0316] The server retrieves furniture supplier and price information from a database for each generated coordination proposal and integrates it into the proposal. Specifically, it issues queries to the APIs of various online stores to obtain appropriate furniture supplier and price information. The input is data on the coordination proposal, supplier, and price information, and the output is the coordination proposal with the supplier and price information added.

[0317] Step 8:

[0318] The terminal receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The display includes furniture layout diagrams, purchase locations, price information, and online purchase links. The input is the coordination suggestion information sent from the server, and the output is the visual information displayed on the terminal.

[0319] Step 9:

[0320] The user selects their preferred furniture from several displayed coordination options and purchases it online. The selection is made through the terminal's interface, and clicking the purchase link initiates the online purchase process. Input is the user's selection information, and output is data confirming the completion of the purchase. The server receives this selection history and stores it to optimize future suggestions.

[0321] The above describes the specific processing flow of this system and the detailed operation at each step.

[0322] (Application Example 2)

[0323] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0324] In today's world, interior design has become incredibly diverse, making it a challenging task to automatically suggest the optimal design based on user preferences and budgets. Furthermore, real-time optimization based on user emotions and moods has not been implemented, and systems that reduce the hassle of purchasing are still lacking. Additionally, virtual environments where users can physically examine furniture online before purchasing are not adequately provided.

[0325] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0326] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for generating coordination proposals using an AI model, means for adding information on where to purchase and the prices of the furniture used in the coordination proposals, means for analyzing the user's emotional state and optimizing the coordination proposals based on those emotions, and means for checking and selecting furniture in a virtual environment. As a result, the user can receive suggestions for the best interior coordination according to their preferences and budget, and furthermore, by optimizing suggestions based on their emotional state, they can obtain coordination proposals that match their mood in real time and purchase furniture while checking it in a virtual environment.

[0327] A "user" refers to a person who uses this system to generate room coordination ideas.

[0328] "Room floor plans and photos" refers to the layout information of a room or images taken by the user.

[0329] A "server" refers to a computer system that receives information sent by users, analyzes it, generates coordination proposals, and provides purchase information.

[0330] "Floor plan" refers to information about the layout of a room, such as its shape and the location of walls, windows, and doors.

[0331] "Preferred taste" refers to the type of interior style that the user desires.

[0332] "Budget" refers to the amount of money a user is willing to pay for an outfit.

[0333] An "AI model" refers to an algorithm used to generate outfit suggestions based on past data and trends.

[0334] "Coordination proposal" refers to the interior design suggestions for a room generated by the server.

[0335] "Emotional state" refers to the user's current emotions, obtained by analyzing their facial expressions and tone of voice.

[0336] "Purchase source" refers to stores or online shops where you can buy the furniture used in the coordination.

[0337] "Price information" refers to the price of the furniture offered by the retailer.

[0338] An "online purchase link" refers to a URL that allows users to purchase the furniture used in the suggested interior design over the internet.

[0339] A "virtual environment" refers to a virtual space that resembles reality, created using computer graphics and other technologies.

[0340] "Emotional analysis means" refers to technology that analyzes a user's emotional state in real time through a camera or microphone.

[0341] This invention is a system that allows users to upload floor plans or photos of their rooms, generate room coordination suggestions using an AI model and emotion engine based on those plans, and then view and select furniture within a virtual environment.

[0342] Hardware and software to use

[0343] Hardware:

[0344] Terminal devices such as smartphones, smart glasses, and head-mounted displays

[0345] Camera and microphone (for the emotion engine)

[0346] software:

[0347] Image recognition algorithm (OpenCV)

[0348] Emotion recognition engine (Microsoft Azure Cognitive Services)

[0349] AI model (AWS SageMaker or Google® Cloud AI)

[0350] Database (MySQL® or Firestore)

[0351] System processing details

[0352] 1. Users upload room floor plans or photos:

[0353] Users upload room layouts and photos to the system using a terminal device. This uploaded data is then sent from the terminal to the server.

[0354] 2. The server analyzes the drawings and photographs:

[0355] The server uses OpenCV to analyze received drawings and photos, automatically recognizing the room layout. This layout information includes the shape of the room, and the locations of walls, windows, and doors.

[0356] 3. The user enters their preferred taste and budget:

[0357] The user accesses a form on their device to enter their preferred taste and budget. The entered information is then sent to the server via the device.

[0358] 4. The server generates a coordination proposal:

[0359] Based on the received taste and budget information, along with the analyzed floor plan information, the server uses an AI model (AWS SageMaker or Google Cloud AI) to generate multiple coordination proposals.

[0360] 5. The emotion engine recognizes the user's emotions:

[0361] Through the camera and microphone built into the device, the emotion engine (Microsoft Azure Cognitive Services) analyzes the user's facial expressions and tone of voice to recognize their current emotional state. This allows the server to provide real-time suggestions tailored to the user's mood.

[0362] 6. The server adds information about the supplier and price:

[0363] The server retrieves information on the source and price of the furniture used in the generated coordination plan from a database (MySQL or Firestore) and adds it to the plan.

[0364] 7. The device displays outfit suggestions:

[0365] The device receives coordination suggestions sent from the server and displays them to the user in a visually easy-to-understand format. The displayed coordination suggestions include furniture placement, purchase locations, price information, and online purchase links.

[0366] 8. View and select furniture within the virtual environment:

[0367] Users can use smart glasses or head-mounted displays to view and select furniture in a virtual environment. The selected and purchased information is updated on the server and used to improve future recommendations.

[0368] Adding specific examples

[0369] For example, suppose a user wants a "Nordic-style" interior and sets a budget of 30,000 yen. When the user uploads photos of their room, the server analyzes the layout, and based on that information and the user's preferences, an AI model generates three different interior design options. An emotion engine analyzes the user's mood and suggests the most suitable option. The user can use smart glasses to walk around the virtual store, view, select, and purchase furniture.

[0370] Example of a prompt

[0371] "Design a system where users input their desired interior style and budget, upload room plans and photos, and then use AI and an emotion engine to suggest the optimal furniture arrangement and design based on that information."

[0372] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0373] Step 1:

[0374] Users upload floor plans and photos of their rooms.

[0375] Input: Room floor plan or photo

[0376] Output: Uploaded drawing or photo data

[0377] Specific operation: Users use devices such as smartphones or tablets to select room layouts and photo files and upload them to the system. This data is sent to the server via the internet.

[0378] Step 2:

[0379] The server receives drawings and photos, analyzes them, and recognizes the floor plan.

[0380] Input: Uploaded drawing or photo data

[0381] Output: Analyzed floor plan information (room shape, wall, window, door locations, etc.)

[0382] Specific operation: The server receives the transmitted image data and analyzes the drawings and photographs using OpenCV. The analysis algorithm recognizes each element of the room (walls, windows, doors, etc.) and structures this as floor plan information.

[0383] Step 3:

[0384] The user enters their preferred taste and budget.

[0385] Input: Preferred style (e.g., "Nordic style"), budget

[0386] Output: Input taste and budget data

[0387] Specific operation: The user accesses a form on their device screen and enters their preferred interior style and estimated budget. This information is sent to the server via the internet.

[0388] Step 4:

[0389] The server uses an AI model to generate multiple coordination options based on the input information and floor plan.

[0390] Input: Taste, budget, floor plan information

[0391] Output: Multiple coordination options

[0392] Specific operation: The server combines the received taste, budget, and floor plan information and uses an AI model to generate the optimal coordination plan. The AI ​​model uses past datasets and trend information to suggest furniture placement and design.

[0393] Step 5:

[0394] The emotion engine recognizes the user's emotions.

[0395] Input: User's facial expressions and tone of voice

[0396] Output: Analyzed user emotional state

[0397] Specific operation: The emotion recognition engine analyzes the user's facial expressions and voice tone through the camera and microphone on the user's device. The results of this analysis are sent to the server.

[0398] Step 6:

[0399] The server optimizes outfit suggestions based on the user's emotional state and adds information about where to purchase the items and their prices.

[0400] Input: Multiple outfit ideas, user's emotional state

[0401] Output: Optimized outfit suggestions, with purchase location and pricing information.

[0402] Specific operation: The server optimizes outfit suggestions based on the received emotional state. Next, it retrieves and adds supplier and price information from the database for each optimized suggestion.

[0403] Step 7:

[0404] The device displays coordination suggestions, allowing users to view furniture within a virtual environment.

[0405] Input: Optimized outfit suggestions, purchase locations, and price information

[0406] Output: User-confirmable outfit suggestions, purchase locations, and price information.

[0407] Specific operation: The device receives data sent from the server and displays it visually to the user. This display uses 3D models or AR technology, allowing users to view furniture in a virtual environment.

[0408] Step 8:

[0409] The user selects a displayed coordination suggestion and purchases the furniture.

[0410] Input: Selection information for coordination options, purchase intent.

[0411] Output: Purchase completion status, information on selected furniture.

[0412] Specific operation: The user selects their preferred coordination plan on their device and presses the purchase button to actually order the furniture via an online purchase link. The selection and purchase history is sent to the server and used for future suggestions.

[0413] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0414] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0415] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0416] [Second Embodiment]

[0417] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0418] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0419] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0420] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0421] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0422] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0423] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0424] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0425] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0426] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0427] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0428] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0429] This invention is a system in which users upload floor plans or photos of their rooms, and an AI model is used to generate room coordination proposals based on these. The system proposes multiple coordination proposals that match the user's preferred style and budget, and also provides information on where to purchase the suggested furniture and their prices. The system's program processing is described below in natural language.

[0430] 1. The user uploads a floor plan or photos of the room.

[0431] Users upload room layouts and photos to the system using their devices. The uploaded layouts and photos are then sent from the device to the server.

[0432] 2. The server analyzes the drawings and photographs.

[0433] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes the room's shape, the location of walls, windows, and doors. This analysis uses image recognition algorithms to extract the accurate layout.

[0434] 3. The user enters their preferred taste and budget.

[0435] The user accesses a form on their device to enter their preferred style (e.g., "resort hotel style") and budget. The information entered by the user is sent to the server via the device.

[0436] 4. The server generates a coordination proposal.

[0437] The server uses an AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model suggests optimal furniture placement and interior design based on past data and trends.

[0438] 5. The server adds information about the supplier and price.

[0439] The server adds information about the purchase locations and prices of the furniture used in the generated coordination proposals. This involves retrieving specific store information and online purchase links from the database and integrating them into the proposal.

[0440] 6. The device displays outfit suggestions.

[0441] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[0442] 7. The user selects and purchases their favorite outfit.

[0443] Users select their preferred outfit from several displayed coordination options. They can easily purchase the selected furniture by clicking the displayed online purchase link. The user's selection history and purchase information are sent from their device to the server and used to optimize future suggestions.

[0444] Specific example

[0445] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. The user can then easily redecorate their room by selecting their favorite design and purchasing the furniture via the displayed link.

[0446] This system allows users to easily coordinate their rooms to their liking and receive support in selecting the best furniture within their budget. The server's AI model constantly learns the latest trends and data, ensuring that the suggested coordination ideas are of high quality and relevant to the times.

[0447] The following describes the processing flow.

[0448] Step 1:

[0449] The user selects room floor plans and photo files using their device and clicks the upload button.

[0450] Step 2:

[0451] The terminal sends the selected drawing or photo file to the server.

[0452] Step 3:

[0453] The server receives drawing and photo files sent from the terminal.

[0454] Step 4:

[0455] The server uses image analysis algorithms to recognize the room layout from the received drawings or photographs. This includes the process of extracting the locations of walls, windows, doors, and furniture.

[0456] Step 5:

[0457] The user accesses a form on their device interface where they can enter their preferred style (e.g., "resort hotel style") and budget.

[0458] Step 6:

[0459] The user enters the required information and clicks the submit button.

[0460] Step 7:

[0461] The device sends user-entered preferences and budget information to the server.

[0462] Step 8:

[0463] The server analyzes the customer's preferred taste and budget information that it receives.

[0464] Step 9:

[0465] Based on floor plan information recognized by the server, as well as the user's preferred style and budget, an AI model is used to generate multiple design options.

[0466] Step 10:

[0467] For each coordinated outfit design generated by the server, the server retrieves information from the database regarding the source and price of the furniture used.

[0468] Step 11:

[0469] The server retrieves the supplier and price information and adds it to each suggested outfit. If online purchase is possible, a link to that purchase is also included.

[0470] Step 12:

[0471] The server sends the outfit suggestions, purchase locations, pricing information, and online purchase links to the device.

[0472] Step 13:

[0473] The terminal receives the suggested outfit from the server and displays it to the user in a visually easy-to-understand format.

[0474] Step 14:

[0475] The user selects their favorite outfit from several displayed outfit options.

[0476] Step 15:

[0477] The user clicks the online purchase link for their chosen interior design and buys the furniture online.

[0478] Step 16:

[0479] The device sends the user's selections and purchase history to the server, which then updates that information.

[0480] (Example 1)

[0481] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0482] Traditionally, it has been time-consuming and laborious for users to coordinate a room to their liking. Furthermore, a lack of knowledge and skills in selecting and arranging appropriate furniture made achieving an ideal coordination difficult. Additionally, choosing the best furniture within a budget was challenging, often limiting options. As a result, users have struggled to achieve a satisfactory room design.

[0483] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0484] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using a generation AI model based on the input information and the layout, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, means for the user to select a displayed coordination proposal and purchase the furniture and update the server accordingly, means for the server to generate prompt messages based on the layout information analyzed by the server and the user's input information and send them to the generation AI model, and means for the server to generate the optimal coordination proposal based on trained data and trend data in the generation AI model. This makes it possible for users to easily create a room coordination that suits their preferences within their budget.

[0485] "A means for users to upload room plans and photos" refers to an interface that allows users to select room plans and photos from their devices and send them to the server.

[0486] "A method for a server to receive and analyze uploaded drawings and photos to recognize the layout of a room" refers to a technology in which a server receives drawings and photos sent by a user and uses an image recognition algorithm to extract layout information such as the shape of the room and the location of walls, windows, and doors.

[0487] "A means for users to input their preferred taste and budget, and a means for the server to receive that information" refers to a system in which users input their preferred interior style and budget through a form displayed on their device, and the server receives the entered information.

[0488] "A means by which a server generates multiple coordination proposals using a generative AI model based on input information and floor plan" refers to a technology in which a server generates multiple interior coordination proposals using a generative AI model that has learned from past data and the latest trends, based on user input information and analyzed floor plan information.

[0489] "A means by which the server adds information on the source and price of furniture used in a coordination plan" refers to a technology in which the server retrieves information on the source and price of furniture used in a generated coordination plan from a database and integrates it into the coordination plan.

[0490] "Means by which a terminal displays coordination suggestions, purchase locations, price information, and online purchase links" refers to an interface that allows the terminal to visually display coordination suggestions, furniture purchase locations, price information, and online purchase links received from the server to the user.

[0491] "A means for a user to select a displayed coordination plan and purchase furniture, and a means for updating that information on a server" refers to a system for a user to select a coordination plan they like from the displayed options and purchase furniture through an online purchase link, and the technology for sending and updating purchase information on a server.

[0492] "A means of generating prompt text based on floor plan information analyzed by the server and user input information, and sending it to a generation AI model" refers to a technology that generates prompt text to be input to a generation AI model based on floor plan information analyzed by the server and user input information, and then inputs that prompt text into the generation AI model.

[0493] "A means by which a server generates optimal coordination proposals based on trained data and trend data from a generated AI model" refers to a technology in which a server uses a generated AI model to consider trained data and the latest trends to generate optimal interior coordination proposals.

[0494] This invention is a system in which a user uploads floor plans or photos of a room, and an AI model is used to generate room coordination suggestions based on that information. The specific steps for implementing this system are shown below.

[0495] Users first upload room layouts and photos using their own devices (computers, smartphones, etc.). This upload is done via an interface using HTML or React. The uploaded layouts and photos are sent from the device to the server, which then receives them.

[0496] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes room shape, wall and window locations, and door positions. Image recognition libraries such as OpenCV and TensorFlow are used for this analysis. The server utilizes image recognition algorithms to extract accurate layouts.

[0497] The user then enters their preferred style (e.g., "resort hotel style") and budget on the on-device interface. A form for this purpose is built using HTML or React, and the information entered by the user is sent to the server via the device.

[0498] The server uses a generative AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. This AI model suggests optimal furniture placement and interior design based on historical and trend data. During the generation process, the server generates prompt messages based on the analyzed layout information and user input, and sends these prompt messages to the generative AI model.

[0499] As a concrete example, if a user requests a "resort hotel style" room design and sets a budget of 50,000 yen, the following prompt message will be generated: "I would like a resort hotel style room design. My budget is 50,000 yen. I have uploaded photos of the room. Based on this information, please generate three optimal design options."

[0500] The server generates optimal coordination suggestions based on data trained by a generation AI model and the latest trend data. Each generated coordination suggestion includes information on where to purchase the furniture used and its price. This allows users to easily create a room coordination that suits their preferences within their budget.

[0501] Ultimately, the device receives the coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links. The user can easily redecorate their room by selecting their favorite from the multiple coordination suggestions displayed and purchasing the furniture through the displayed links.

[0502] As described above, this system provides an effective means for users to easily coordinate their rooms and choose the optimal furniture within their budget.

[0503] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0504] Step 1:

[0505] Users upload room layouts and photos. Specifically, users use their devices to click the "Upload" button and select room layout and photo files. Then, they press the "Submit" button, and the selected files are sent from their device to the server. The input is the room layout and photos provided by the user, and the output is the image files sent to the server.

[0506] Step 2:

[0507] The server analyzes the received drawings and photographs. At this stage, the received image files are analyzed using image recognition libraries such as OpenCV or TensorFlow. The input is the image file sent to the server, and the output is the analyzed room layout information (room shape, location of walls, windows, doors, etc.). The server triggers the "start image recognition" process, analyzing the image pixel by pixel to extract room features.

[0508] Step 3:

[0509] The user inputs their preferred style and budget. Specifically, the user enters a style such as "resort hotel style" and a budget such as "50,000 yen" into a form provided on the device's interface. The form used for this is built with HTML or React. The input is the user's preferred style and budget, and the output is the style and budget information sent from the device to the server.

[0510] Step 4:

[0511] The server uses a generating AI model to create multiple coordination proposals based on the user's preferred taste, budget information, and analyzed floor plan information. The input is the user's taste information, budget information, and floor plan information, and the output is multiple coordination proposals generated by the generating AI model. The server combines the analysis results and the user's wishes to generate a prompt message and input it into the AI ​​model. For example, "I would like a resort hotel-style room coordination. My budget is 50,000 yen. I have uploaded photos of the room. Based on this information, please generate three optimal coordination proposals."

[0512] Step 5:

[0513] The server adds information about the source and price of the furniture used in the generated coordination proposals. The server retrieves source and price information from the database and integrates it into each coordination proposal. The input is the generated coordination proposal, and the output is the coordination proposal with added source and price information. Here, the server queries for specific store information and online shop links and integrates them into the proposal content.

[0514] Step 6:

[0515] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The input is the coordination suggestion information sent from the server, and the output is a detailed furniture layout diagram, purchase location, price information, and online purchase links displayed to the user. This information is displayed to the user on the device using HTML or React.

[0516] Step 7:

[0517] The user selects a displayed coordination plan and purchases the furniture. Specifically, the user selects a preferred coordination plan on their device and clicks the displayed online purchase link to complete the purchase process. The input is the user's selection of coordination plans and purchase process, and the output is the selection history and purchase information sent from the device to the server. The server stores this information in a database and uses it to optimize future suggestions.

[0518] (Application Example 1)

[0519] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0520] In modern life, interior design is an important element for many people, but there is a lack of easy and efficient ways to do it. Furthermore, the process of users selecting interior items that suit their taste and purchasing them within their budget is complex and cumbersome. Moreover, there is currently no system utilizing appropriate AI models to provide design suggestions, and these problems need to be addressed.

[0521] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0522] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using an AI model based on the input information and the layout, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, means for the user to select a displayed coordination proposal and purchase the furniture and update the server accordingly, means for visually displaying the interior design proposals generated by the AI ​​model via a smartphone app to support the purchase, means for displaying suggested content based on the user's preferences, and means for checking the where to purchase and the price of the furniture included in the interior design proposals online and performing the purchase procedure directly via the link. This makes it possible for users to easily check, select, and purchase interior coordination that suits their preferences and budget.

[0523] A "user" is an individual or corporation that uses this system to create room interior design plans or purchase furniture.

[0524] "Room floor plans and photos" refer to data containing visual information that users upload to the system to identify the layout and characteristics of a room.

[0525] A "server" is a central processing unit that analyzes drawings and photos received from users, generates coordination proposals, and also provides information on furniture suppliers and pricing.

[0526] "Analysis" refers to the process by which a server recognizes the layout and interior features of a room based on drawings and photographs.

[0527] "Preferred style and budget" refers to the interior style and amount of money that users can spend, which serve as the basis for generating room coordination suggestions.

[0528] An "AI model" refers to an artificial intelligence algorithm or computational model that uses input information and floor plan information to determine the optimal interior layout and furniture selection.

[0529] A "coordination plan" is a proposal generated by an AI model that shows the arrangement and selection of interior furnishings for a specific room.

[0530] "Purchase location and price information" refers to information about where the furniture included in the coordination plan can be purchased and how much it costs.

[0531] An "online purchase link" is a direct reference URL to a website or online store where users can immediately purchase the suggested furniture.

[0532] A "smartphone app" is application software that users can install to check room interior design ideas or purchase furniture.

[0533] "Selection history" refers to the record data of the coordination ideas and furniture that the user has selected and purchased so far.

[0534] "Optimization" refers to the process of adjusting the next suggestion to better match the user's needs, based on their preferences and past selection history.

[0535] This invention is a system in which a user uploads floor plans or photos of a room, and an AI model is used to generate room coordination suggestions based on that information. Specifically, the invention can be implemented through the following steps.

[0536] Hardware and software to be used

[0537] This system is implemented using the following hardware and software.

[0538] Smartphone: A device for applications that the user installs.

[0539] Server: Central processing unit that performs image analysis and inference processing using AI models.

[0540] OpenCV: A library for servers to perform image processing.

[0541] TensorFlow: A machine learning library for performing inference processing on AI models.

[0542] Database module: Manages furniture purchase information and pricing.

[0543] Program processing

[0544] The processes performed at each step of this system are as follows:

[0545] 1. Users upload room floor plans or photos:

[0546] Users upload room layouts and photos using a smartphone app. The uploaded layouts and photos are then sent from the smartphone to the server.

[0547] 2. The server analyzes the drawings and photos:

[0548] The server uses the OpenCV library to analyze the received drawings and photographs. The analysis automatically recognizes the room layout, extracting information such as the room's shape and the locations of walls, windows, and doors.

[0549] 3. The user enters their preferred taste and budget:

[0550] Users enter their preferred style (e.g., "resort hotel style") and budget on a smartphone app. The information entered by the user is sent to the server via the smartphone.

[0551] 4. The server generates a coordination proposal:

[0552] The server uses an AI model powered by TensorFlow to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model then suggests the optimal furniture placement and interior design based on past data and trends.

[0553] 5. The server adds the supplier and price information:

[0554] The server retrieves and adds information about the purchase locations and prices of the furniture used in the generated coordination plan from a database module. The process then integrates specific store information and online purchase links into the coordination plan.

[0555] 6. Your smartphone will display outfit suggestions:

[0556] The smartphone app receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[0557] 7. The user selects and purchases their favorite outfit:

[0558] Users select their favorite from several displayed coordination options and purchase the furniture via a link displayed through the smartphone app. The purchase information of the selected furniture is updated on the server and used to optimize future suggestions.

[0559] Specific example

[0560] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by information on where to buy it, its price, and an online purchase link. The user can then select their favorite room design and easily redecorate their room by purchasing the furniture via the link displayed on the smartphone app.

[0561] Example of a prompt

[0562] The following input prompts can be set for the generative AI model.

[0563] Prompt: Please provide images and floor plan information for the room. Then, generate the best interior design proposal to match the specified budget and style.

[0564] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0565] Step 1:

[0566] Users upload floor plans and photos of their rooms. Users use their smartphones to take photos and upload them to the server via the application. The input here is the floor plan and photo data sent from the user's device, while the output is the raw image data received on the server side.

[0567] Step 2:

[0568] The server analyzes drawings and photographs to recognize the room layout. The server uses OpenCV to extract structural information about the room within the image and automatically identifies the layout. The input here is the image data received in step 1, and the output is layout information such as the shape of the room and the positions of walls, windows, and doors. Specifically, the server resizes the image and extracts feature points to recognize the room layout.

[0569] Step 3:

[0570] The user enters their preferred taste and budget. The user sets their taste (e.g., "modern" or "classic") and budget within the application and sends this information to the server. The input here is the taste and budget information entered by the user, while the output is the preference and budget data received on the server side.

[0571] Step 4:

[0572] The server generates multiple coordination options using an AI model. Based on the received taste and budget information, and the analyzed floor plan information, the server creates coordination options from the AI ​​model using TensorFlow. The input here is the output data from steps 2 and 3, and the output is the multiple coordination options that have been generated. Specifically, the input data is passed to the AI ​​model, which calculates the optimal furniture placement and interior design.

[0573] Step 5:

[0574] The server adds information about the source and price of the furniture used in the coordination proposal. The server queries the database module for source and price information for the generated coordination proposal and adds the source information and price to each coordination. The input here is the coordination proposal generated in step 4, and the output is data with source and price information for the furniture associated with each coordination added.

[0575] Step 6:

[0576] The device displays coordination suggestions, purchase locations, price information, and online purchase links. The smartphone app visually displays coordination suggestions based on the information received from the server. The input here is the data generated in step 5, and the output is a display of coordination suggestions that the user can view. Specifically, the app screen displays a furniture layout diagram and purchase links for each item.

[0577] Step 7:

[0578] The user selects a displayed coordination suggestion and purchases the furniture. The user chooses their favorite from several displayed coordination suggestions and clicks the online purchase link via the smartphone app to purchase the furniture. The input here is the coordination suggestion displayed in step 6, and the output is information about the purchased items added to the shopping cart. Specifically, the user clicks the displayed link and completes the purchase procedure and payment. The purchase information of the selected furniture is updated on the server and used to optimize future suggestions.

[0579] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0580] This invention is a system in which a user uploads floor plans or photos of a room, and based on that, an AI model and emotion engine are used to generate room coordination suggestions. The system proposes multiple coordination suggestions that match the user's preferred style and budget, and also provides information on where to purchase the suggested furniture and their prices. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, the system optimizes the coordination suggestions according to the user's mood and past emotional data. The system's program processing is described below in natural language.

[0581] 1. The user uploads a floor plan or photos of the room.

[0582] Users upload room layouts and photos to the system using their devices. The uploaded layouts and photos are then sent from the device to the server.

[0583] 2. The server analyzes the drawings and photographs.

[0584] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes the room's shape, the location of walls, windows, and doors. This analysis uses image recognition algorithms to extract the accurate layout.

[0585] 3. The user enters their preferred taste and budget.

[0586] The user accesses a form on their device to enter their preferred style (e.g., "resort hotel style") and budget. The information entered by the user is sent to the server via the device.

[0587] 4. The server generates a coordination proposal.

[0588] The server uses an AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model suggests optimal furniture placement and interior design based on past data and trends.

[0589] 5. The emotion engine recognizes the user's emotions.

[0590] The emotion engine analyzes the user's facial expressions and voice tone through the camera and microphone built into the device to recognize their current emotional state. This allows the server to provide real-time suggestions tailored to the user's mood.

[0591] 6. The server adds information about the supplier and price.

[0592] The server retrieves information on the source and price of the furniture used in the generated coordination plan from its database. This includes retrieving and integrating specific store information and online purchase links.

[0593] 7. The device displays outfit suggestions.

[0594] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[0595] 8. The user selects and purchases their favorite outfit.

[0596] Users select their preferred outfit from several displayed options. In addition, an emotion engine evaluates the user's emotional response to their selection and adjusts the suggestions as needed. Selected furniture can be easily purchased by clicking the displayed online purchase link. The user's selection history and purchase information are sent from the device to the server and used to optimize future suggestions.

[0597] Specific example

[0598] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. Then, an emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, prioritizing and displaying the room design option that best suits their current mood. The user can easily redecorate their room by selecting a specific room design option and purchasing the furniture through the displayed link.

[0599] This system allows users to easily decorate their rooms to their liking, and through real-time suggestion optimization powered by an emotion engine, they can receive optimal suggestions that match their mood. Furthermore, the server's AI model constantly learns the latest trends and data, ensuring that the suggested decorating ideas are of high quality and relevant to the times.

[0600] The following describes the processing flow.

[0601] Step 1:

[0602] The user uses their device to select room floor plans or photo files and clicks the upload button.

[0603] Step 2:

[0604] The terminal sends the selected drawing or photo file to the server.

[0605] Step 3:

[0606] The server receives drawing and photo files sent from the terminal.

[0607] Step 4:

[0608] The server uses image analysis algorithms to recognize the room layout from the received drawings or photographs. This includes the process of extracting the locations of walls, windows, doors, and furniture.

[0609] Step 5:

[0610] The user accesses a form on their device interface to enter their preferred style (e.g., "resort hotel style") and budget.

[0611] Step 6:

[0612] The user enters the required information and clicks the submit button.

[0613] Step 7:

[0614] The device sends user-entered preferences and budget information to the server.

[0615] Step 8:

[0616] The server receives information about the user's preferred tastes and budget.

[0617] Step 9:

[0618] Based on floor plan information recognized by the server, as well as the user's preferred style and budget, an AI model is used to generate multiple design options.

[0619] Step 10:

[0620] For each generated design proposal, the server retrieves information on where to purchase the furniture and its price from the database. This includes obtaining specific store information and online purchase links.

[0621] Step 11:

[0622] The server retrieves the supplier and price information and adds it to each suggested outfit. If online purchase is possible, a link to that purchase is also included.

[0623] Step 12:

[0624] The server sends the outfit suggestions, purchase locations, pricing information, and online purchase links to the device.

[0625] Step 13:

[0626] The terminal receives coordination proposal information sent from the server and displays it to the user in a visually easy-to-understand format.

[0627] Step 14:

[0628] The user selects their favorite outfit from the displayed outfit suggestions.

[0629] Step 15:

[0630] The emotion engine analyzes the user's facial expressions and voice tone through the device's camera and microphone to recognize their current emotional state.

[0631] Step 16:

[0632] The server prioritizes displaying outfit suggestions that best suit the user's current mood, based on their emotional state.

[0633] Step 17:

[0634] The user clicks the online purchase link for the coordination plan they ultimately selected and buys the furniture online.

[0635] Step 18:

[0636] The device sends the user's selections and purchase history to a server, which is then recorded to optimize future suggestions.

[0637] (Example 2)

[0638] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0639] Conventional room coordination suggestion systems failed to consider the user's emotions and moods, making it difficult to increase user satisfaction. Furthermore, information on furniture suppliers and pricing based on the suggested coordination plans was scarce, making it difficult for users to make consistent purchase decisions without hassle. Additionally, there was a challenge in generating and presenting coordination plans that matched the user's preferences and budget in real time.

[0640] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0641] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using an AI model based on the input information and the layout, means for an emotion engine to recognize the user's emotions and optimize the proposals based on that information, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, and means for the user to select a displayed coordination proposal, purchase the furniture, and update the server accordingly. This makes it possible to provide more personalized coordination proposals that take into account the user's emotions and mood, to consistently provide information on where to purchase and the price of the furniture, and to improve user satisfaction.

[0642] - A "user" is the entity that uses this system to coordinate a room.

[0643] A "device" refers to an electronic device used by a user, such as a smartphone or personal computer.

[0644] A "server" is the central processing unit of a system, a computer that receives, analyzes, generates, and transmits data.

[0645] A "drawing" refers to a design plan or floor plan that shows the layout of a room.

[0646] A "photograph" is a still image that captures the current state of the room.

[0647] "Analysis" refers to the process of extracting floor plans and room features from uploaded drawings and photographs.

[0648] "Preferred taste" refers to the type of interior style or design that the user desires.

[0649] "Budget" refers to the maximum amount of money a user is willing to spend on coordinating an outfit.

[0650] An "AI model" is artificial intelligence that learns from past data and trends and generates outfit suggestions based on user input.

[0651] An "emotion engine" is software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[0652] "Purchase location" refers to the stores or online shops where the suggested furniture can be purchased.

[0653] "Price information" refers to information regarding the price of the proposed furniture.

[0654] An "online purchase link" is a URL that allows users to purchase furniture via the internet.

[0655] A "coordination plan" refers to furniture arrangements and interior designs proposed based on the user's preferred style, budget, and room layout.

[0656] "Selection history" refers to information about the coordination ideas and furniture that the user has previously selected in the system.

[0657] "Suggestion optimization" is the process of improving suggestions based on the user's selection history and emotional state.

[0658] This invention relates to a system in which a user uploads floor plans or photos of a room, and based on that, an AI model and an emotion engine are used to generate room coordination suggestions. The system allows the user to input their preferred style and budget, and then proposes multiple coordination suggestions that fit the budget, also providing information on where to purchase the suggested furniture and their prices. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, the system optimizes the coordination suggestions according to the user's mood and past emotional data.

[0659] Hardware and software

[0660] This system is implemented using the following hardware and software:

[0661] Server: This is a central processing unit that receives, analyzes, generates, and transmits data. Specifically, it uses cloud computing services such as Amazon Web Services (AWS) or Microsoft Azure.

[0662] Device: A smartphone or computer used by the user. These devices provide the user interface and allow for data uploading and display.

[0663] AI Model: Generative AI models such as Generative Adversarial Networks (GANs) are used to generate coordination proposals.

[0664] Emotion Engine: Utilizes Amazon Rekognition and Microsoft Azure's Emotion API to analyze the user's facial expressions and voice tone, and recognize their emotional state.

[0665] Image recognition algorithm: Using OpenCV and TensorFlow, analyze room layouts from uploaded drawings and photos.

[0666] Data processing and data calculation

[0667] 1. Uploading and receiving data:

[0668] Users upload room layouts and photos from their devices. This data is sent to the server via the internet.

[0669] 2. Image analysis:

[0670] The server stores the received drawings and photos and uses image recognition algorithms to analyze the room layout. This automatically recognizes the placement of walls, doors, and windows.

[0671] 3. Enter user information:

[0672] The user enters their preferred taste and budget into a form on their device. This information is then sent from the device to the server.

[0673] 4. Generating coordination proposals:

[0674] The server uses an AI model to generate multiple coordination options based on user input and analyzed floor plan data.

[0675] 5. Emotion recognition:

[0676] The emotion engine analyzes the user's facial expressions and voice tone through the device's camera and microphone, recognizing their emotional state in real time. This information is sent to a server and used to optimize suggestions.

[0677] 6. Add purchase information:

[0678] The server retrieves furniture supplier and price information from the database for each generated coordination plan and integrates it into the plan.

[0679] 7. Display of outfit suggestions:

[0680] The terminal receives coordination proposal information sent from the server and displays it to the user in a visually easy-to-understand format. The display includes furniture layout diagrams, purchase locations, price information, and online purchase links.

[0681] 8. Selection and Purchase:

[0682] Users can purchase furniture online based on their selected coordination plan. Purchase information is sent to the server and stored to optimize future suggestions.

[0683] Specific example

[0684] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. Then, an emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, prioritizing and displaying the room design option that best suits their current mood. The user can easily redecorate their room by selecting a specific room design option and purchasing the furniture through the displayed link.

[0685] Example of a prompt

[0686] Based on the following room photos and floor plan information, please propose three resort-hotel-style living room design ideas within a budget of 50,000 yen. [Attach room photos and floor plan information]

[0687] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0688] Step 1:

[0689] The user uploads room plans and photos using their device. Specifically, the user opens a file selection dialog on their device and selects the room plans and photos. The input here is the room plans and photos, which are sent from the device to the server as an HTTP POST request. The output is the data of the plans and photos transferred to the server.

[0690] Step 2:

[0691] The server analyzes the drawings and photos it receives. Specifically, it temporarily stores the received files and extracts room layout information using an image recognition algorithm (e.g., OpenCV or TensorFlow). The input is the uploaded drawing or photo file, and the output is room layout information (location of walls, doors, windows, etc.).

[0692] Step 3:

[0693] The user enters their preferred style and budget through the terminal's interface. The user enters their style (e.g., "resort hotel style") and budget into the form and presses the "Submit" button. The input consists of style and budget information, which is sent from the terminal to the server. The output is the submitted style and budget data.

[0694] Step 4:

[0695] Based on the server's received preferences, budget information, and analyzed floor plan data, an AI model is used to generate multiple coordination proposals. Specifically, generative AI models such as Generative Adversarial Networks (GANs) are used. The inputs are preferences, budget information, and floor plan data, and the output is multiple coordination proposals.

[0696] Step 5:

[0697] The emotion engine analyzes the user's facial expressions and voice tone through the device's built-in camera and microphone, recognizing their emotional state in real time. The input is data on the user's facial expressions and voice tone, which the emotion engine analyzes. The output is data on the user's current emotional state.

[0698] Step 6:

[0699] The server receives emotional state data from the emotion engine and optimizes the suggested outfit combinations. Specifically, this data is used to provide suggestions that are tailored to the user's emotional state in real time. The input is emotional state data, and the output is an outfit combination optimized according to that emotion.

[0700] Step 7:

[0701] The server retrieves furniture supplier and price information from a database for each generated coordination proposal and integrates it into the proposal. Specifically, it issues queries to the APIs of various online stores to obtain appropriate furniture supplier and price information. The input is data on the coordination proposal, supplier, and price information, and the output is the coordination proposal with the supplier and price information added.

[0702] Step 8:

[0703] The terminal receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The display includes furniture layout diagrams, purchase locations, price information, and online purchase links. The input is the coordination suggestion information sent from the server, and the output is the visual information displayed on the terminal.

[0704] Step 9:

[0705] The user selects their preferred furniture from several displayed coordination options and purchases it online. The selection is made through the terminal's interface, and clicking the purchase link initiates the online purchase process. Input is the user's selection information, and output is data confirming the completion of the purchase. The server receives this selection history and stores it to optimize future suggestions.

[0706] The above describes the specific processing flow of this system and the detailed operation at each step.

[0707] (Application Example 2)

[0708] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0709] In today's world, interior design has become incredibly diverse, making it a challenging task to automatically suggest the optimal design based on user preferences and budgets. Furthermore, real-time optimization based on user emotions and moods has not been implemented, and systems that reduce the hassle of purchasing are still lacking. Additionally, virtual environments where users can physically examine furniture online before purchasing are not adequately provided.

[0710] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0711] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for generating coordination proposals using an AI model, means for adding information on where to purchase and the prices of the furniture used in the coordination proposals, means for analyzing the user's emotional state and optimizing the coordination proposals based on those emotions, and means for checking and selecting furniture in a virtual environment. As a result, the user can receive suggestions for the best interior coordination according to their preferences and budget, and furthermore, by optimizing suggestions based on their emotional state, they can obtain coordination proposals that match their mood in real time and purchase furniture while checking it in a virtual environment.

[0712] A "user" refers to a person who uses this system to generate room coordination ideas.

[0713] "Room floor plans and photos" refers to the layout information of a room or images taken by the user.

[0714] A "server" refers to a computer system that receives information sent by users, analyzes it, generates coordination proposals, and provides purchase information.

[0715] "Floor plan" refers to information about the layout of a room, such as its shape and the location of walls, windows, and doors.

[0716] "Preferred taste" refers to the type of interior style that the user desires.

[0717] "Budget" refers to the amount of money a user is willing to pay for an outfit.

[0718] An "AI model" refers to an algorithm used to generate outfit suggestions based on past data and trends.

[0719] "Coordination proposal" refers to the interior design suggestions for a room generated by the server.

[0720] "Emotional state" refers to the user's current emotions, obtained by analyzing their facial expressions and tone of voice.

[0721] "Purchase source" refers to stores or online shops where you can buy the furniture used in the coordination.

[0722] "Price information" refers to the price of the furniture offered by the retailer.

[0723] An "online purchase link" refers to a URL that allows users to purchase the furniture used in the suggested interior design over the internet.

[0724] A "virtual environment" refers to a virtual space that resembles reality, created using computer graphics and other technologies.

[0725] "Emotional analysis means" refers to technology that analyzes a user's emotional state in real time through a camera or microphone.

[0726] This invention is a system that allows users to upload floor plans or photos of their rooms, generate room coordination suggestions using an AI model and emotion engine based on those plans, and then view and select furniture within a virtual environment.

[0727] Hardware and software to use

[0728] Hardware:

[0729] Terminal devices such as smartphones, smart glasses, and head-mounted displays

[0730] Camera and microphone (for the emotion engine)

[0731] software:

[0732] Image recognition algorithm (OpenCV)

[0733] Emotion recognition engine (Microsoft Azure Cognitive Services)

[0734] AI model (AWS SageMaker or Google Cloud AI)

[0735] Database (MySQL or Firestore)

[0736] System processing details

[0737] 1. Users upload room floor plans or photos:

[0738] Users upload room layouts and photos to the system using a terminal device. This uploaded data is then sent from the terminal to the server.

[0739] 2. The server analyzes the drawings and photographs:

[0740] The server uses OpenCV to analyze received drawings and photos, automatically recognizing the room layout. This layout information includes the shape of the room, and the locations of walls, windows, and doors.

[0741] 3. The user enters their preferred taste and budget:

[0742] The user accesses a form on their device to enter their preferred taste and budget. The entered information is then sent to the server via the device.

[0743] 4. The server generates a coordination proposal:

[0744] Based on the received taste and budget information, along with the analyzed floor plan information, the server uses an AI model (AWS SageMaker or Google Cloud AI) to generate multiple coordination proposals.

[0745] 5. The emotion engine recognizes the user's emotions:

[0746] Through the camera and microphone built into the device, the emotion engine (Microsoft Azure Cognitive Services) analyzes the user's facial expressions and tone of voice to recognize their current emotional state. This allows the server to provide real-time suggestions tailored to the user's mood.

[0747] 6. The server adds information about the supplier and price:

[0748] The server retrieves information on the source and price of the furniture used in the generated coordination plan from a database (MySQL or Firestore) and adds it to the plan.

[0749] 7. The device displays outfit suggestions:

[0750] The device receives coordination suggestions sent from the server and displays them to the user in a visually easy-to-understand format. The displayed coordination suggestions include furniture placement, purchase locations, price information, and online purchase links.

[0751] 8. View and select furniture within the virtual environment:

[0752] Users can use smart glasses or head-mounted displays to view and select furniture in a virtual environment. The selected and purchased information is updated on the server and used to improve future recommendations.

[0753] Adding specific examples

[0754] For example, suppose a user wants a "Nordic-style" interior and sets a budget of 30,000 yen. When the user uploads photos of their room, the server analyzes the layout, and based on that information and the user's preferences, an AI model generates three different interior design options. An emotion engine analyzes the user's mood and suggests the most suitable option. The user can use smart glasses to walk around the virtual store, view, select, and purchase furniture.

[0755] Example of a prompt

[0756] "Design a system where users input their desired interior style and budget, upload room plans and photos, and then use AI and an emotion engine to suggest the optimal furniture arrangement and design based on that information."

[0757] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0758] Step 1:

[0759] Users upload floor plans and photos of their rooms.

[0760] Input: Room floor plan or photo

[0761] Output: Uploaded drawing or photo data

[0762] Specific operation: Users use devices such as smartphones or tablets to select room layouts and photo files and upload them to the system. This data is sent to the server via the internet.

[0763] Step 2:

[0764] The server receives drawings and photos, analyzes them, and recognizes the floor plan.

[0765] Input: Uploaded drawing or photo data

[0766] Output: Analyzed floor plan information (room shape, wall, window, door locations, etc.)

[0767] Specific operation: The server receives the transmitted image data and analyzes the drawings and photographs using OpenCV. The analysis algorithm recognizes each element of the room (walls, windows, doors, etc.) and structures this as floor plan information.

[0768] Step 3:

[0769] The user enters their preferred taste and budget.

[0770] Input: Preferred style (e.g., "Nordic style"), budget

[0771] Output: Input taste and budget data

[0772] Specific operation: The user accesses a form on their device screen and enters their preferred interior style and estimated budget. This information is sent to the server via the internet.

[0773] Step 4:

[0774] The server uses an AI model to generate multiple coordination options based on the input information and floor plan.

[0775] Input: Taste, budget, floor plan information

[0776] Output: Multiple coordination options

[0777] Specific operation: The server combines the received taste, budget, and floor plan information and uses an AI model to generate the optimal coordination plan. The AI ​​model uses past datasets and trend information to suggest furniture placement and design.

[0778] Step 5:

[0779] The emotion engine recognizes the user's emotions.

[0780] Input: User's facial expressions and tone of voice

[0781] Output: Analyzed user emotional state

[0782] Specific operation: The emotion recognition engine analyzes the user's facial expressions and voice tone through the camera and microphone on the user's device. The results of this analysis are sent to the server.

[0783] Step 6:

[0784] The server optimizes outfit suggestions based on the user's emotional state and adds information about where to purchase the items and their prices.

[0785] Input: Multiple outfit ideas, user's emotional state

[0786] Output: Optimized outfit suggestions, with purchase location and pricing information.

[0787] Specific operation: The server optimizes outfit suggestions based on the received emotional state. Next, it retrieves and adds supplier and price information from the database for each optimized suggestion.

[0788] Step 7:

[0789] The device displays coordination suggestions, allowing users to view furniture within a virtual environment.

[0790] Input: Optimized outfit suggestions, purchase locations, and price information

[0791] Output: User-confirmable outfit suggestions, purchase locations, and price information.

[0792] Specific operation: The device receives data sent from the server and displays it visually to the user. This display uses 3D models or AR technology, allowing users to view furniture in a virtual environment.

[0793] Step 8:

[0794] The user selects a displayed coordination suggestion and purchases the furniture.

[0795] Input: Selection information for coordination options, purchase intent.

[0796] Output: Purchase completion status, information on selected furniture.

[0797] Specific operation: The user selects their preferred coordination plan on their device and presses the purchase button to actually order the furniture via an online purchase link. The selection and purchase history is sent to the server and used for future suggestions.

[0798] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0799] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0800] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0801] [Third Embodiment]

[0802] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0803] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0804] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0805] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0806] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0807] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0808] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0809] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0810] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0811] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0812] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0813] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0814] This invention is a system in which users upload floor plans or photos of their rooms, and an AI model is used to generate room coordination proposals based on these. The system proposes multiple coordination proposals that match the user's preferred style and budget, and also provides information on where to purchase the suggested furniture and their prices. The system's program processing is described below in natural language.

[0815] 1. The user uploads a floor plan or photos of the room.

[0816] Users upload room layouts and photos to the system using their devices. The uploaded layouts and photos are then sent from the device to the server.

[0817] 2. The server analyzes the drawings and photographs.

[0818] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes the room's shape, the location of walls, windows, and doors. This analysis uses image recognition algorithms to extract the accurate layout.

[0819] 3. The user enters their preferred taste and budget.

[0820] The user accesses a form on their device to enter their preferred style (e.g., "resort hotel style") and budget. The information entered by the user is sent to the server via the device.

[0821] 4. The server generates a coordination proposal.

[0822] The server uses an AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model suggests optimal furniture placement and interior design based on past data and trends.

[0823] 5. The server adds information about the supplier and price.

[0824] The server adds information about the purchase locations and prices of the furniture used in the generated coordination proposals. This involves retrieving specific store information and online purchase links from the database and integrating them into the proposal.

[0825] 6. The device displays outfit suggestions.

[0826] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[0827] 7. The user selects and purchases their favorite outfit.

[0828] Users select their preferred outfit from several displayed coordination options. They can easily purchase the selected furniture by clicking the displayed online purchase link. The user's selection history and purchase information are sent from their device to the server and used to optimize future suggestions.

[0829] Specific example

[0830] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. The user can then easily redecorate their room by selecting their favorite design and purchasing the furniture via the displayed link.

[0831] This system allows users to easily coordinate their rooms to their liking and receive support in selecting the best furniture within their budget. The server's AI model constantly learns the latest trends and data, ensuring that the suggested coordination ideas are of high quality and relevant to the times.

[0832] The following describes the processing flow.

[0833] Step 1:

[0834] The user selects room floor plans and photo files using their device and clicks the upload button.

[0835] Step 2:

[0836] The terminal sends the selected drawing or photo file to the server.

[0837] Step 3:

[0838] The server receives drawing and photo files sent from the terminal.

[0839] Step 4:

[0840] The server uses image analysis algorithms to recognize the room layout from the received drawings or photographs. This includes the process of extracting the locations of walls, windows, doors, and furniture.

[0841] Step 5:

[0842] The user accesses a form on their device interface where they can enter their preferred style (e.g., "resort hotel style") and budget.

[0843] Step 6:

[0844] The user enters the required information and clicks the submit button.

[0845] Step 7:

[0846] The device sends user-entered preferences and budget information to the server.

[0847] Step 8:

[0848] The server analyzes the customer's preferred taste and budget information that it receives.

[0849] Step 9:

[0850] Based on floor plan information recognized by the server, as well as the user's preferred style and budget, an AI model is used to generate multiple design options.

[0851] Step 10:

[0852] For each coordinated outfit design generated by the server, the server retrieves information from the database regarding the source and price of the furniture used.

[0853] Step 11:

[0854] The server retrieves the supplier and price information and adds it to each suggested outfit. If online purchase is possible, a link to that purchase is also included.

[0855] Step 12:

[0856] The server sends the outfit suggestions, purchase locations, pricing information, and online purchase links to the device.

[0857] Step 13:

[0858] The terminal receives the suggested outfit from the server and displays it to the user in a visually easy-to-understand format.

[0859] Step 14:

[0860] The user selects their favorite outfit from several displayed outfit options.

[0861] Step 15:

[0862] The user clicks the online purchase link for their chosen interior design and buys the furniture online.

[0863] Step 16:

[0864] The device sends the user's selections and purchase history to the server, which then updates that information.

[0865] (Example 1)

[0866] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0867] Traditionally, it has been time-consuming and laborious for users to coordinate a room to their liking. Furthermore, a lack of knowledge and skills in selecting and arranging appropriate furniture made achieving an ideal coordination difficult. Additionally, choosing the best furniture within a budget was challenging, often limiting options. As a result, users have struggled to achieve a satisfactory room design.

[0868] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0869] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using a generation AI model based on the input information and the layout, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, means for the user to select a displayed coordination proposal and purchase the furniture and update the server accordingly, means for the server to generate prompt messages based on the layout information analyzed by the server and the user's input information and send them to the generation AI model, and means for the server to generate the optimal coordination proposal based on trained data and trend data in the generation AI model. This makes it possible for users to easily create a room coordination that suits their preferences within their budget.

[0870] "A means for users to upload room plans and photos" refers to an interface that allows users to select room plans and photos from their devices and send them to the server.

[0871] "A method for a server to receive and analyze uploaded drawings and photos to recognize the layout of a room" refers to a technology in which a server receives drawings and photos sent by a user and uses an image recognition algorithm to extract layout information such as the shape of the room and the location of walls, windows, and doors.

[0872] "A means for users to input their preferred taste and budget, and a means for the server to receive that information" refers to a system in which users input their preferred interior style and budget through a form displayed on their device, and the server receives the entered information.

[0873] "A means by which a server generates multiple coordination proposals using a generative AI model based on input information and floor plan" refers to a technology in which a server generates multiple interior coordination proposals using a generative AI model that has learned from past data and the latest trends, based on user input information and analyzed floor plan information.

[0874] "A means by which the server adds information on the source and price of furniture used in a coordination plan" refers to a technology in which the server retrieves information on the source and price of furniture used in a generated coordination plan from a database and integrates it into the coordination plan.

[0875] "Means by which a terminal displays coordination suggestions, purchase locations, price information, and online purchase links" refers to an interface that allows the terminal to visually display coordination suggestions, furniture purchase locations, price information, and online purchase links received from the server to the user.

[0876] "A means for a user to select a displayed coordination plan and purchase furniture, and a means for updating that information on a server" refers to a system for a user to select a coordination plan they like from the displayed options and purchase furniture through an online purchase link, and the technology for sending and updating purchase information on a server.

[0877] "A means of generating prompt text based on floor plan information analyzed by the server and user input information, and sending it to a generation AI model" refers to a technology that generates prompt text to be input to a generation AI model based on floor plan information analyzed by the server and user input information, and then inputs that prompt text into the generation AI model.

[0878] "A means by which a server generates optimal coordination proposals based on trained data and trend data from a generated AI model" refers to a technology in which a server uses a generated AI model to consider trained data and the latest trends to generate optimal interior coordination proposals.

[0879] This invention is a system in which a user uploads floor plans or photos of a room, and an AI model is used to generate room coordination suggestions based on that information. The specific steps for implementing this system are shown below.

[0880] Users first upload room layouts and photos using their own devices (computers, smartphones, etc.). This upload is done via an interface using HTML or React. The uploaded layouts and photos are sent from the device to the server, which then receives them.

[0881] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes room shape, wall and window locations, and door positions. Image recognition libraries such as OpenCV and TensorFlow are used for this analysis. The server utilizes image recognition algorithms to extract accurate layouts.

[0882] The user then enters their preferred style (e.g., "resort hotel style") and budget on the on-device interface. A form for this purpose is built using HTML or React, and the information entered by the user is sent to the server via the device.

[0883] The server uses a generative AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. This AI model suggests optimal furniture placement and interior design based on historical and trend data. During the generation process, the server generates prompt messages based on the analyzed layout information and user input, and sends these prompt messages to the generative AI model.

[0884] As a concrete example, if a user requests a "resort hotel style" room design and sets a budget of 50,000 yen, the following prompt message will be generated: "I would like a resort hotel style room design. My budget is 50,000 yen. I have uploaded photos of the room. Based on this information, please generate three optimal design options."

[0885] The server generates optimal coordination suggestions based on data trained by a generation AI model and the latest trend data. Each generated coordination suggestion includes information on where to purchase the furniture used and its price. This allows users to easily create a room coordination that suits their preferences within their budget.

[0886] Ultimately, the device receives the coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links. The user can easily redecorate their room by selecting their favorite from the multiple coordination suggestions displayed and purchasing the furniture through the displayed links.

[0887] As described above, this system provides an effective means for users to easily coordinate their rooms and choose the optimal furniture within their budget.

[0888] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0889] Step 1:

[0890] Users upload room layouts and photos. Specifically, users use their devices to click the "Upload" button and select room layout and photo files. Then, they press the "Submit" button, and the selected files are sent from their device to the server. The input is the room layout and photos provided by the user, and the output is the image files sent to the server.

[0891] Step 2:

[0892] The server analyzes the received drawings and photographs. At this stage, the received image files are analyzed using image recognition libraries such as OpenCV or TensorFlow. The input is the image file sent to the server, and the output is the analyzed room layout information (room shape, location of walls, windows, doors, etc.). The server triggers the "start image recognition" process, analyzing the image pixel by pixel to extract room features.

[0893] Step 3:

[0894] The user inputs their preferred style and budget. Specifically, the user enters a style such as "resort hotel style" and a budget such as "50,000 yen" into a form provided on the device's interface. The form used for this is built with HTML or React. The input is the user's preferred style and budget, and the output is the style and budget information sent from the device to the server.

[0895] Step 4:

[0896] The server uses a generating AI model to create multiple coordination proposals based on the user's preferred taste, budget information, and analyzed floor plan information. The input is the user's taste information, budget information, and floor plan information, and the output is multiple coordination proposals generated by the generating AI model. The server combines the analysis results and the user's wishes to generate a prompt message and input it into the AI ​​model. For example, "I would like a resort hotel-style room coordination. My budget is 50,000 yen. I have uploaded photos of the room. Based on this information, please generate three optimal coordination proposals."

[0897] Step 5:

[0898] The server adds information about the source and price of the furniture used in the generated coordination proposals. The server retrieves source and price information from the database and integrates it into each coordination proposal. The input is the generated coordination proposal, and the output is the coordination proposal with added source and price information. Here, the server queries for specific store information and online shop links and integrates them into the proposal content.

[0899] Step 6:

[0900] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The input is the coordination suggestion information sent from the server, and the output is a detailed furniture layout diagram, purchase location, price information, and online purchase links displayed to the user. This information is displayed to the user on the device using HTML or React.

[0901] Step 7:

[0902] The user selects a displayed coordination plan and purchases the furniture. Specifically, the user selects a preferred coordination plan on their device and clicks the displayed online purchase link to complete the purchase process. The input is the user's selection of coordination plans and purchase process, and the output is the selection history and purchase information sent from the device to the server. The server stores this information in a database and uses it to optimize future suggestions.

[0903] (Application Example 1)

[0904] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0905] In modern life, interior design is an important element for many people, but there is a lack of easy and efficient ways to do it. Furthermore, the process of users selecting interior items that suit their taste and purchasing them within their budget is complex and cumbersome. Moreover, there is currently no system utilizing appropriate AI models to provide design suggestions, and these problems need to be addressed.

[0906] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0907] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using an AI model based on the input information and the layout, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, means for the user to select a displayed coordination proposal and purchase the furniture and update the server accordingly, means for visually displaying the interior design proposals generated by the AI ​​model via a smartphone app to support the purchase, means for displaying suggested content based on the user's preferences, and means for checking the where to purchase and the price of the furniture included in the interior design proposals online and performing the purchase procedure directly via the link. This makes it possible for users to easily check, select, and purchase interior coordination that suits their preferences and budget.

[0908] A "user" is an individual or corporation that uses this system to create room interior design plans or purchase furniture.

[0909] "Room floor plans and photos" refer to data containing visual information that users upload to the system to identify the layout and characteristics of a room.

[0910] A "server" is a central processing unit that analyzes drawings and photos received from users, generates coordination proposals, and also provides information on furniture suppliers and pricing.

[0911] "Analysis" refers to the process by which a server recognizes the layout and interior features of a room based on drawings and photographs.

[0912] "Preferred style and budget" refers to the interior style and amount of money that users can spend, which serve as the basis for generating room coordination suggestions.

[0913] An "AI model" refers to an artificial intelligence algorithm or computational model that uses input information and floor plan information to determine the optimal interior layout and furniture selection.

[0914] A "coordination plan" is a proposal generated by an AI model that shows the arrangement and selection of interior furnishings for a specific room.

[0915] "Purchase location and price information" refers to information about where the furniture included in the coordination plan can be purchased and how much it costs.

[0916] An "online purchase link" is a direct reference URL to a website or online store where users can immediately purchase the suggested furniture.

[0917] A "smartphone app" is application software that users can install to check room interior design ideas or purchase furniture.

[0918] "Selection history" refers to the record data of the coordination ideas and furniture that the user has selected and purchased so far.

[0919] "Optimization" refers to the process of adjusting the next suggestion to better match the user's needs, based on their preferences and past selection history.

[0920] This invention is a system in which a user uploads floor plans or photos of a room, and an AI model is used to generate room coordination suggestions based on that information. Specifically, the invention can be implemented through the following steps.

[0921] Hardware and software to be used

[0922] This system is implemented using the following hardware and software.

[0923] Smartphone: A device for applications that the user installs.

[0924] Server: Central processing unit that performs image analysis and inference processing using AI models.

[0925] OpenCV: A library for servers to perform image processing.

[0926] TensorFlow: A machine learning library for performing inference processing on AI models.

[0927] Database module: Manages furniture purchase information and pricing.

[0928] Program processing

[0929] The processes performed at each step of this system are as follows:

[0930] 1. Users upload room floor plans or photos:

[0931] Users upload room layouts and photos using a smartphone app. The uploaded layouts and photos are then sent from the smartphone to the server.

[0932] 2. The server analyzes the drawings and photos:

[0933] The server uses the OpenCV library to analyze the received drawings and photographs. The analysis automatically recognizes the room layout, extracting information such as the room's shape and the locations of walls, windows, and doors.

[0934] 3. The user enters their preferred taste and budget:

[0935] Users enter their preferred style (e.g., "resort hotel style") and budget on a smartphone app. The information entered by the user is sent to the server via the smartphone.

[0936] 4. The server generates a coordination proposal:

[0937] The server uses an AI model powered by TensorFlow to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model then suggests the optimal furniture placement and interior design based on past data and trends.

[0938] 5. The server adds the supplier and price information:

[0939] The server retrieves and adds information about the purchase locations and prices of the furniture used in the generated coordination plan from a database module. The process then integrates specific store information and online purchase links into the coordination plan.

[0940] 6. Your smartphone will display outfit suggestions:

[0941] The smartphone app receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[0942] 7. The user selects and purchases their favorite outfit:

[0943] Users select their favorite from several displayed coordination options and purchase the furniture via a link displayed through the smartphone app. The purchase information of the selected furniture is updated on the server and used to optimize future suggestions.

[0944] Specific example

[0945] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by information on where to buy it, its price, and an online purchase link. The user can then select their favorite room design and easily redecorate their room by purchasing the furniture via the link displayed on the smartphone app.

[0946] Example of a prompt

[0947] The following input prompts can be set for the generative AI model.

[0948] Prompt: Please provide images and floor plan information for the room. Then, generate the best interior design proposal to match the specified budget and style.

[0949] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0950] Step 1:

[0951] Users upload floor plans and photos of their rooms. Users use their smartphones to take photos and upload them to the server via the application. The input here is the floor plan and photo data sent from the user's device, while the output is the raw image data received on the server side.

[0952] Step 2:

[0953] The server analyzes drawings and photographs to recognize the room layout. The server uses OpenCV to extract structural information about the room within the image and automatically identifies the layout. The input here is the image data received in step 1, and the output is layout information such as the shape of the room and the positions of walls, windows, and doors. Specifically, the server resizes the image and extracts feature points to recognize the room layout.

[0954] Step 3:

[0955] The user enters their preferred taste and budget. The user sets their taste (e.g., "modern" or "classic") and budget within the application and sends this information to the server. The input here is the taste and budget information entered by the user, while the output is the preference and budget data received on the server side.

[0956] Step 4:

[0957] The server generates multiple coordination options using an AI model. Based on the received taste and budget information, and the analyzed floor plan information, the server creates coordination options from the AI ​​model using TensorFlow. The input here is the output data from steps 2 and 3, and the output is the multiple coordination options that have been generated. Specifically, the input data is passed to the AI ​​model, which calculates the optimal furniture placement and interior design.

[0958] Step 5:

[0959] The server adds information about the source and price of the furniture used in the coordination proposal. The server queries the database module for source and price information for the generated coordination proposal and adds the source information and price to each coordination. The input here is the coordination proposal generated in step 4, and the output is data with source and price information for the furniture associated with each coordination added.

[0960] Step 6:

[0961] The device displays coordination suggestions, purchase locations, price information, and online purchase links. The smartphone app visually displays coordination suggestions based on the information received from the server. The input here is the data generated in step 5, and the output is a display of coordination suggestions that the user can view. Specifically, the app screen displays a furniture layout diagram and purchase links for each item.

[0962] Step 7:

[0963] The user selects a displayed coordination suggestion and purchases the furniture. The user chooses their favorite from several displayed coordination suggestions and clicks the online purchase link via the smartphone app to purchase the furniture. The input here is the coordination suggestion displayed in step 6, and the output is information about the purchased items added to the shopping cart. Specifically, the user clicks the displayed link and completes the purchase procedure and payment. The purchase information of the selected furniture is updated on the server and used to optimize future suggestions.

[0964] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0965] This invention is a system in which a user uploads floor plans or photos of a room, and based on that, an AI model and emotion engine are used to generate room coordination suggestions. The system proposes multiple coordination suggestions that match the user's preferred style and budget, and also provides information on where to purchase the suggested furniture and their prices. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, the system optimizes the coordination suggestions according to the user's mood and past emotional data. The system's program processing is described below in natural language.

[0966] 1. The user uploads a floor plan or photos of the room.

[0967] Users upload room layouts and photos to the system using their devices. The uploaded layouts and photos are then sent from the device to the server.

[0968] 2. The server analyzes the drawings and photographs.

[0969] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes the room's shape, the location of walls, windows, and doors. This analysis uses image recognition algorithms to extract the accurate layout.

[0970] 3. The user enters their preferred taste and budget.

[0971] The user accesses a form on their device to enter their preferred style (e.g., "resort hotel style") and budget. The information entered by the user is sent to the server via the device.

[0972] 4. The server generates a coordination proposal.

[0973] The server uses an AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model suggests optimal furniture placement and interior design based on past data and trends.

[0974] 5. The emotion engine recognizes the user's emotions.

[0975] The emotion engine analyzes the user's facial expressions and voice tone through the camera and microphone built into the device to recognize their current emotional state. This allows the server to provide real-time suggestions tailored to the user's mood.

[0976] 6. The server adds information about the supplier and price.

[0977] The server retrieves information on the source and price of the furniture used in the generated coordination plan from its database. This includes retrieving and integrating specific store information and online purchase links.

[0978] 7. The device displays outfit suggestions.

[0979] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[0980] 8. The user selects and purchases their favorite outfit.

[0981] Users select their preferred outfit from several displayed options. In addition, an emotion engine evaluates the user's emotional response to their selection and adjusts the suggestions as needed. Selected furniture can be easily purchased by clicking the displayed online purchase link. The user's selection history and purchase information are sent from the device to the server and used to optimize future suggestions.

[0982] Specific example

[0983] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. Then, an emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, prioritizing and displaying the room design option that best suits their current mood. The user can easily redecorate their room by selecting a specific room design option and purchasing the furniture through the displayed link.

[0984] This system allows users to easily decorate their rooms to their liking, and through real-time suggestion optimization powered by an emotion engine, they can receive optimal suggestions that match their mood. Furthermore, the server's AI model constantly learns the latest trends and data, ensuring that the suggested decorating ideas are of high quality and relevant to the times.

[0985] The following describes the processing flow.

[0986] Step 1:

[0987] The user uses their device to select room floor plans or photo files and clicks the upload button.

[0988] Step 2:

[0989] The terminal sends the selected drawing or photo file to the server.

[0990] Step 3:

[0991] The server receives drawing and photo files sent from the terminal.

[0992] Step 4:

[0993] The server uses image analysis algorithms to recognize the room layout from the received drawings or photographs. This includes the process of extracting the locations of walls, windows, doors, and furniture.

[0994] Step 5:

[0995] The user accesses a form on their device interface to enter their preferred style (e.g., "resort hotel style") and budget.

[0996] Step 6:

[0997] The user enters the required information and clicks the submit button.

[0998] Step 7:

[0999] The device sends user-entered preferences and budget information to the server.

[1000] Step 8:

[1001] The server receives information about the user's preferred tastes and budget.

[1002] Step 9:

[1003] Based on floor plan information recognized by the server, as well as the user's preferred style and budget, an AI model is used to generate multiple design options.

[1004] Step 10:

[1005] For each generated design proposal, the server retrieves information on where to purchase the furniture and its price from the database. This includes obtaining specific store information and online purchase links.

[1006] Step 11:

[1007] The server retrieves the supplier and price information and adds it to each suggested outfit. If online purchase is possible, a link to that purchase is also included.

[1008] Step 12:

[1009] The server sends the outfit suggestions, purchase locations, pricing information, and online purchase links to the device.

[1010] Step 13:

[1011] The terminal receives coordination proposal information sent from the server and displays it to the user in a visually easy-to-understand format.

[1012] Step 14:

[1013] The user selects their favorite outfit from the displayed outfit suggestions.

[1014] Step 15:

[1015] The emotion engine analyzes the user's facial expressions and voice tone through the device's camera and microphone to recognize their current emotional state.

[1016] Step 16:

[1017] The server prioritizes displaying outfit suggestions that best suit the user's current mood, based on their emotional state.

[1018] Step 17:

[1019] The user clicks the online purchase link for the coordination plan they ultimately selected and buys the furniture online.

[1020] Step 18:

[1021] The device sends the user's selections and purchase history to a server, which is then recorded to optimize future suggestions.

[1022] (Example 2)

[1023] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1024] Conventional room coordination suggestion systems failed to consider the user's emotions and moods, making it difficult to increase user satisfaction. Furthermore, information on furniture suppliers and pricing based on the suggested coordination plans was scarce, making it difficult for users to make consistent purchase decisions without hassle. Additionally, there was a challenge in generating and presenting coordination plans that matched the user's preferences and budget in real time.

[1025] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1026] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using an AI model based on the input information and the layout, means for an emotion engine to recognize the user's emotions and optimize the proposals based on that information, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, and means for the user to select a displayed coordination proposal, purchase the furniture, and update the server accordingly. This makes it possible to provide more personalized coordination proposals that take into account the user's emotions and mood, to consistently provide information on where to purchase and the price of the furniture, and to improve user satisfaction.

[1027] - A "user" is the entity that uses this system to coordinate a room.

[1028] A "device" refers to an electronic device used by a user, such as a smartphone or personal computer.

[1029] A "server" is the central processing unit of a system, a computer that receives, analyzes, generates, and transmits data.

[1030] A "drawing" refers to a design plan or floor plan that shows the layout of a room.

[1031] A "photograph" is a still image that captures the current state of the room.

[1032] "Analysis" refers to the process of extracting floor plans and room features from uploaded drawings and photographs.

[1033] "Preferred taste" refers to the type of interior style or design that the user desires.

[1034] "Budget" refers to the maximum amount of money a user is willing to spend on coordinating an outfit.

[1035] An "AI model" is artificial intelligence that learns from past data and trends and generates outfit suggestions based on user input.

[1036] An "emotion engine" is software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[1037] "Purchase location" refers to the stores or online shops where the suggested furniture can be purchased.

[1038] "Price information" refers to information regarding the price of the proposed furniture.

[1039] An "online purchase link" is a URL that allows users to purchase furniture via the internet.

[1040] A "coordination plan" refers to furniture arrangements and interior designs proposed based on the user's preferred style, budget, and room layout.

[1041] "Selection history" refers to information about the coordination ideas and furniture that the user has previously selected in the system.

[1042] "Suggestion optimization" is the process of improving suggestions based on the user's selection history and emotional state.

[1043] This invention relates to a system in which a user uploads floor plans or photos of a room, and based on that, an AI model and an emotion engine are used to generate room coordination suggestions. The system allows the user to input their preferred style and budget, and then proposes multiple coordination suggestions that fit the budget, also providing information on where to purchase the suggested furniture and their prices. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, the system optimizes the coordination suggestions according to the user's mood and past emotional data.

[1044] Hardware and software

[1045] This system is implemented using the following hardware and software:

[1046] Server: This is a central processing unit that receives, analyzes, generates, and transmits data. Specifically, it uses cloud computing services such as Amazon Web Services (AWS) or Microsoft Azure.

[1047] Device: A smartphone or computer used by the user. These devices provide the user interface and allow for data uploading and display.

[1048] AI Model: Generative AI models such as Generative Adversarial Networks (GANs) are used to generate coordination proposals.

[1049] Emotion Engine: Utilizes Amazon Rekognition and Microsoft Azure's Emotion API to analyze the user's facial expressions and voice tone, and recognize their emotional state.

[1050] Image recognition algorithm: Using OpenCV and TensorFlow, analyze room layouts from uploaded drawings and photos.

[1051] Data processing and data calculation

[1052] 1. Uploading and receiving data:

[1053] Users upload room layouts and photos from their devices. This data is sent to the server via the internet.

[1054] 2. Image analysis:

[1055] The server stores the received drawings and photos and uses image recognition algorithms to analyze the room layout. This automatically recognizes the placement of walls, doors, and windows.

[1056] 3. Enter user information:

[1057] The user enters their preferred taste and budget into a form on their device. This information is then sent from the device to the server.

[1058] 4. Generating coordination proposals:

[1059] The server uses an AI model to generate multiple coordination options based on user input and analyzed floor plan data.

[1060] 5. Emotion recognition:

[1061] The emotion engine analyzes the user's facial expressions and voice tone through the device's camera and microphone, recognizing their emotional state in real time. This information is sent to a server and used to optimize suggestions.

[1062] 6. Add purchase information:

[1063] The server retrieves furniture supplier and price information from the database for each generated coordination plan and integrates it into the plan.

[1064] 7. Display of outfit suggestions:

[1065] The terminal receives coordination proposal information sent from the server and displays it to the user in a visually easy-to-understand format. The display includes furniture layout diagrams, purchase locations, price information, and online purchase links.

[1066] 8. Selection and Purchase:

[1067] Users can purchase furniture online based on their selected coordination plan. Purchase information is sent to the server and stored to optimize future suggestions.

[1068] Specific example

[1069] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. Then, an emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, prioritizing and displaying the room design option that best suits their current mood. The user can easily redecorate their room by selecting a specific room design option and purchasing the furniture through the displayed link.

[1070] Example of a prompt

[1071] Based on the following room photos and floor plan information, please propose three resort-hotel-style living room design ideas within a budget of 50,000 yen. [Attach room photos and floor plan information]

[1072] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1073] Step 1:

[1074] The user uploads room plans and photos using their device. Specifically, the user opens a file selection dialog on their device and selects the room plans and photos. The input here is the room plans and photos, which are sent from the device to the server as an HTTP POST request. The output is the data of the plans and photos transferred to the server.

[1075] Step 2:

[1076] The server analyzes the drawings and photos it receives. Specifically, it temporarily stores the received files and extracts room layout information using an image recognition algorithm (e.g., OpenCV or TensorFlow). The input is the uploaded drawing or photo file, and the output is room layout information (location of walls, doors, windows, etc.).

[1077] Step 3:

[1078] The user enters their preferred style and budget through the terminal's interface. The user enters their style (e.g., "resort hotel style") and budget into the form and presses the "Submit" button. The input consists of style and budget information, which is sent from the terminal to the server. The output is the submitted style and budget data.

[1079] Step 4:

[1080] Based on the server's received preferences, budget information, and analyzed floor plan data, an AI model is used to generate multiple coordination proposals. Specifically, generative AI models such as Generative Adversarial Networks (GANs) are used. The inputs are preferences, budget information, and floor plan data, and the output is multiple coordination proposals.

[1081] Step 5:

[1082] The emotion engine analyzes the user's facial expressions and voice tone through the device's built-in camera and microphone, recognizing their emotional state in real time. The input is data on the user's facial expressions and voice tone, which the emotion engine analyzes. The output is data on the user's current emotional state.

[1083] Step 6:

[1084] The server receives emotional state data from the emotion engine and optimizes the suggested outfit combinations. Specifically, this data is used to provide suggestions that are tailored to the user's emotional state in real time. The input is emotional state data, and the output is an outfit combination optimized according to that emotion.

[1085] Step 7:

[1086] The server retrieves furniture supplier and price information from a database for each generated coordination proposal and integrates it into the proposal. Specifically, it issues queries to the APIs of various online stores to obtain appropriate furniture supplier and price information. The input is data on the coordination proposal, supplier, and price information, and the output is the coordination proposal with the supplier and price information added.

[1087] Step 8:

[1088] The terminal receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The display includes furniture layout diagrams, purchase locations, price information, and online purchase links. The input is the coordination suggestion information sent from the server, and the output is the visual information displayed on the terminal.

[1089] Step 9:

[1090] The user selects their preferred furniture from several displayed coordination options and purchases it online. The selection is made through the terminal's interface, and clicking the purchase link initiates the online purchase process. Input is the user's selection information, and output is data confirming the completion of the purchase. The server receives this selection history and stores it to optimize future suggestions.

[1091] The above describes the specific processing flow of this system and the detailed operation at each step.

[1092] (Application Example 2)

[1093] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1094] In today's world, interior design has become incredibly diverse, making it a challenging task to automatically suggest the optimal design based on user preferences and budgets. Furthermore, real-time optimization based on user emotions and moods has not been implemented, and systems that reduce the hassle of purchasing are still lacking. Additionally, virtual environments where users can physically examine furniture online before purchasing are not adequately provided.

[1095] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1096] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for generating coordination proposals using an AI model, means for adding information on where to purchase and the prices of the furniture used in the coordination proposals, means for analyzing the user's emotional state and optimizing the coordination proposals based on those emotions, and means for checking and selecting furniture in a virtual environment. As a result, the user can receive suggestions for the best interior coordination according to their preferences and budget, and furthermore, by optimizing suggestions based on their emotional state, they can obtain coordination proposals that match their mood in real time and purchase furniture while checking it in a virtual environment.

[1097] A "user" refers to a person who uses this system to generate room coordination ideas.

[1098] "Room floor plans and photos" refers to the layout information of a room or images taken by the user.

[1099] A "server" refers to a computer system that receives information sent by users, analyzes it, generates coordination proposals, and provides purchase information.

[1100] "Floor plan" refers to information about the layout of a room, such as its shape and the location of walls, windows, and doors.

[1101] "Preferred taste" refers to the type of interior style that the user desires.

[1102] "Budget" refers to the amount of money a user is willing to pay for an outfit.

[1103] An "AI model" refers to an algorithm used to generate outfit suggestions based on past data and trends.

[1104] "Coordination proposal" refers to the interior design suggestions for a room generated by the server.

[1105] "Emotional state" refers to the user's current emotions, obtained by analyzing their facial expressions and tone of voice.

[1106] "Purchase source" refers to stores or online shops where you can buy the furniture used in the coordination.

[1107] "Price information" refers to the price of the furniture offered by the retailer.

[1108] An "online purchase link" refers to a URL that allows users to purchase the furniture used in the suggested interior design over the internet.

[1109] A "virtual environment" refers to a virtual space that resembles reality, created using computer graphics and other technologies.

[1110] "Emotional analysis means" refers to technology that analyzes a user's emotional state in real time through a camera or microphone.

[1111] This invention is a system that allows users to upload floor plans or photos of their rooms, generate room coordination suggestions using an AI model and emotion engine based on those plans, and then view and select furniture within a virtual environment.

[1112] Hardware and software to use

[1113] Hardware:

[1114] Terminal devices such as smartphones, smart glasses, and head-mounted displays

[1115] Camera and microphone (for the emotion engine)

[1116] software:

[1117] Image recognition algorithm (OpenCV)

[1118] Emotion recognition engine (Microsoft Azure Cognitive Services)

[1119] AI model (AWS SageMaker or Google Cloud AI)

[1120] Database (MySQL or Firestore)

[1121] System processing details

[1122] 1. Users upload room floor plans or photos:

[1123] Users upload room layouts and photos to the system using a terminal device. This uploaded data is then sent from the terminal to the server.

[1124] 2. The server analyzes the drawings and photographs:

[1125] The server uses OpenCV to analyze received drawings and photos, automatically recognizing the room layout. This layout information includes the shape of the room, and the locations of walls, windows, and doors.

[1126] 3. The user enters their preferred taste and budget:

[1127] The user accesses a form on their device to enter their preferred taste and budget. The entered information is then sent to the server via the device.

[1128] 4. The server generates a coordination proposal:

[1129] Based on the received taste and budget information, along with the analyzed floor plan information, the server uses an AI model (AWS SageMaker or Google Cloud AI) to generate multiple coordination proposals.

[1130] 5. The emotion engine recognizes the user's emotions:

[1131] Through the camera and microphone built into the device, the emotion engine (Microsoft Azure Cognitive Services) analyzes the user's facial expressions and tone of voice to recognize their current emotional state. This allows the server to provide real-time suggestions tailored to the user's mood.

[1132] 6. The server adds information about the supplier and price:

[1133] The server retrieves information on the source and price of the furniture used in the generated coordination plan from a database (MySQL or Firestore) and adds it to the plan.

[1134] 7. The device displays outfit suggestions:

[1135] The device receives coordination suggestions sent from the server and displays them to the user in a visually easy-to-understand format. The displayed coordination suggestions include furniture placement, purchase locations, price information, and online purchase links.

[1136] 8. View and select furniture within the virtual environment:

[1137] Users can use smart glasses or head-mounted displays to view and select furniture in a virtual environment. The selected and purchased information is updated on the server and used to improve future recommendations.

[1138] Adding specific examples

[1139] For example, suppose a user wants a "Nordic-style" interior and sets a budget of 30,000 yen. When the user uploads photos of their room, the server analyzes the layout, and based on that information and the user's preferences, an AI model generates three different interior design options. An emotion engine analyzes the user's mood and suggests the most suitable option. The user can use smart glasses to walk around the virtual store, view, select, and purchase furniture.

[1140] Example of a prompt

[1141] "Design a system where users input their desired interior style and budget, upload room plans and photos, and then use AI and an emotion engine to suggest the optimal furniture arrangement and design based on that information."

[1142] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1143] Step 1:

[1144] Users upload floor plans and photos of their rooms.

[1145] Input: Room floor plan or photo

[1146] Output: Uploaded drawing or photo data

[1147] Specific operation: Users use devices such as smartphones or tablets to select room layouts and photo files and upload them to the system. This data is sent to the server via the internet.

[1148] Step 2:

[1149] The server receives drawings and photos, analyzes them, and recognizes the floor plan.

[1150] Input: Uploaded drawing or photo data

[1151] Output: Analyzed floor plan information (room shape, wall, window, door locations, etc.)

[1152] Specific operation: The server receives the transmitted image data and analyzes the drawings and photographs using OpenCV. The analysis algorithm recognizes each element of the room (walls, windows, doors, etc.) and structures this as floor plan information.

[1153] Step 3:

[1154] The user enters their preferred taste and budget.

[1155] Input: Preferred style (e.g., "Nordic style"), budget

[1156] Output: Input taste and budget data

[1157] Specific operation: The user accesses a form on their device screen and enters their preferred interior style and estimated budget. This information is sent to the server via the internet.

[1158] Step 4:

[1159] The server uses an AI model to generate multiple coordination options based on the input information and floor plan.

[1160] Input: Taste, budget, floor plan information

[1161] Output: Multiple coordination options

[1162] Specific operation: The server combines the received taste, budget, and floor plan information and uses an AI model to generate the optimal coordination plan. The AI ​​model uses past datasets and trend information to suggest furniture placement and design.

[1163] Step 5:

[1164] The emotion engine recognizes the user's emotions.

[1165] Input: User's facial expressions and tone of voice

[1166] Output: Analyzed user emotional state

[1167] Specific operation: The emotion recognition engine analyzes the user's facial expressions and voice tone through the camera and microphone on the user's device. The results of this analysis are sent to the server.

[1168] Step 6:

[1169] The server optimizes outfit suggestions based on the user's emotional state and adds information about where to purchase the items and their prices.

[1170] Input: Multiple outfit ideas, user's emotional state

[1171] Output: Optimized outfit suggestions, with purchase location and pricing information.

[1172] Specific operation: The server optimizes outfit suggestions based on the received emotional state. Next, it retrieves and adds supplier and price information from the database for each optimized suggestion.

[1173] Step 7:

[1174] The device displays coordination suggestions, allowing users to view furniture within a virtual environment.

[1175] Input: Optimized outfit suggestions, purchase locations, and price information

[1176] Output: User-confirmable outfit suggestions, purchase locations, and price information.

[1177] Specific operation: The device receives data sent from the server and displays it visually to the user. This display uses 3D models or AR technology, allowing users to view furniture in a virtual environment.

[1178] Step 8:

[1179] The user selects a displayed coordination suggestion and purchases the furniture.

[1180] Input: Selection information for coordination options, purchase intent.

[1181] Output: Purchase completion status, information on selected furniture.

[1182] Specific operation: The user selects their preferred coordination plan on their device and presses the purchase button to actually order the furniture via an online purchase link. The selection and purchase history is sent to the server and used for future suggestions.

[1183] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1184] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1185] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1186] [Fourth Embodiment]

[1187] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1188] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1189] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1190] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1191] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1192] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1193] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1194] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1195] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1196] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1197] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1198] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1199] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1200] This invention is a system in which users upload floor plans or photos of their rooms, and an AI model is used to generate room coordination proposals based on these. The system proposes multiple coordination proposals that match the user's preferred style and budget, and also provides information on where to purchase the suggested furniture and their prices. The system's program processing is described below in natural language.

[1201] 1. The user uploads a floor plan or photos of the room.

[1202] Users upload room layouts and photos to the system using their devices. The uploaded layouts and photos are then sent from the device to the server.

[1203] 2. The server analyzes the drawings and photographs.

[1204] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes the room's shape, the location of walls, windows, and doors. This analysis uses image recognition algorithms to extract the accurate layout.

[1205] 3. The user enters their preferred taste and budget.

[1206] The user accesses a form on their device to enter their preferred style (e.g., "resort hotel style") and budget. The information entered by the user is sent to the server via the device.

[1207] 4. The server generates a coordination proposal.

[1208] The server uses an AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model suggests optimal furniture placement and interior design based on past data and trends.

[1209] 5. The server adds information about the supplier and price.

[1210] The server adds information about the purchase locations and prices of the furniture used in the generated coordination proposals. This involves retrieving specific store information and online purchase links from the database and integrating them into the proposal.

[1211] 6. The device displays outfit suggestions.

[1212] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[1213] 7. The user selects and purchases their favorite outfit.

[1214] Users select their preferred outfit from several displayed coordination options. They can easily purchase the selected furniture by clicking the displayed online purchase link. The user's selection history and purchase information are sent from their device to the server and used to optimize future suggestions.

[1215] Specific example

[1216] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. The user can then easily redecorate their room by selecting their favorite design and purchasing the furniture via the displayed link.

[1217] This system allows users to easily coordinate their rooms to their liking and receive support in selecting the best furniture within their budget. The server's AI model constantly learns the latest trends and data, ensuring that the suggested coordination ideas are of high quality and relevant to the times.

[1218] The following describes the processing flow.

[1219] Step 1:

[1220] The user selects room floor plans and photo files using their device and clicks the upload button.

[1221] Step 2:

[1222] The terminal sends the selected drawing or photo file to the server.

[1223] Step 3:

[1224] The server receives drawing and photo files sent from the terminal.

[1225] Step 4:

[1226] The server uses image analysis algorithms to recognize the room layout from the received drawings or photographs. This includes the process of extracting the locations of walls, windows, doors, and furniture.

[1227] Step 5:

[1228] The user accesses a form on their device interface where they can enter their preferred style (e.g., "resort hotel style") and budget.

[1229] Step 6:

[1230] The user enters the required information and clicks the submit button.

[1231] Step 7:

[1232] The device sends user-entered preferences and budget information to the server.

[1233] Step 8:

[1234] The server analyzes the customer's preferred taste and budget information that it receives.

[1235] Step 9:

[1236] Based on floor plan information recognized by the server, as well as the user's preferred style and budget, an AI model is used to generate multiple design options.

[1237] Step 10:

[1238] For each coordinated outfit design generated by the server, the server retrieves information from the database regarding the source and price of the furniture used.

[1239] Step 11:

[1240] The server retrieves the supplier and price information and adds it to each suggested outfit. If online purchase is possible, a link to that purchase is also included.

[1241] Step 12:

[1242] The server sends the outfit suggestions, purchase locations, pricing information, and online purchase links to the device.

[1243] Step 13:

[1244] The terminal receives the suggested outfit from the server and displays it to the user in a visually easy-to-understand format.

[1245] Step 14:

[1246] The user selects their favorite outfit from several displayed outfit options.

[1247] Step 15:

[1248] The user clicks the online purchase link for their chosen interior design and buys the furniture online.

[1249] Step 16:

[1250] The device sends the user's selections and purchase history to the server, which then updates that information.

[1251] (Example 1)

[1252] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1253] Traditionally, it has been time-consuming and laborious for users to coordinate a room to their liking. Furthermore, a lack of knowledge and skills in selecting and arranging appropriate furniture made achieving an ideal coordination difficult. Additionally, choosing the best furniture within a budget was challenging, often limiting options. As a result, users have struggled to achieve a satisfactory room design.

[1254] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1255] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using a generation AI model based on the input information and the layout, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, means for the user to select a displayed coordination proposal and purchase the furniture and update the server accordingly, means for the server to generate prompt messages based on the layout information analyzed by the server and the user's input information and send them to the generation AI model, and means for the server to generate the optimal coordination proposal based on trained data and trend data in the generation AI model. This makes it possible for users to easily create a room coordination that suits their preferences within their budget.

[1256] "A means for users to upload room plans and photos" refers to an interface that allows users to select room plans and photos from their devices and send them to the server.

[1257] "A method for a server to receive and analyze uploaded drawings and photos to recognize the layout of a room" refers to a technology in which a server receives drawings and photos sent by a user and uses an image recognition algorithm to extract layout information such as the shape of the room and the location of walls, windows, and doors.

[1258] "A means for users to input their preferred taste and budget, and a means for the server to receive that information" refers to a system in which users input their preferred interior style and budget through a form displayed on their device, and the server receives the entered information.

[1259] "A means by which a server generates multiple coordination proposals using a generative AI model based on input information and floor plan" refers to a technology in which a server generates multiple interior coordination proposals using a generative AI model that has learned from past data and the latest trends, based on user input information and analyzed floor plan information.

[1260] "A means by which the server adds information on the source and price of furniture used in a coordination plan" refers to a technology in which the server retrieves information on the source and price of furniture used in a generated coordination plan from a database and integrates it into the coordination plan.

[1261] "Means by which a terminal displays coordination suggestions, purchase locations, price information, and online purchase links" refers to an interface that allows the terminal to visually display coordination suggestions, furniture purchase locations, price information, and online purchase links received from the server to the user.

[1262] "A means for a user to select a displayed coordination plan and purchase furniture, and a means for updating that information on a server" refers to a system for a user to select a coordination plan they like from the displayed options and purchase furniture through an online purchase link, and the technology for sending and updating purchase information on a server.

[1263] "A means of generating prompt text based on floor plan information analyzed by the server and user input information, and sending it to a generation AI model" refers to a technology that generates prompt text to be input to a generation AI model based on floor plan information analyzed by the server and user input information, and then inputs that prompt text into the generation AI model.

[1264] "A means by which a server generates optimal coordination proposals based on trained data and trend data from a generated AI model" refers to a technology in which a server uses a generated AI model to consider trained data and the latest trends to generate optimal interior coordination proposals.

[1265] This invention is a system in which a user uploads floor plans or photos of a room, and an AI model is used to generate room coordination suggestions based on that information. The specific steps for implementing this system are shown below.

[1266] Users first upload room layouts and photos using their own devices (computers, smartphones, etc.). This upload is done via an interface using HTML or React. The uploaded layouts and photos are sent from the device to the server, which then receives them.

[1267] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes room shape, wall and window locations, and door positions. Image recognition libraries such as OpenCV and TensorFlow are used for this analysis. The server utilizes image recognition algorithms to extract accurate layouts.

[1268] The user then enters their preferred style (e.g., "resort hotel style") and budget on the on-device interface. A form for this purpose is built using HTML or React, and the information entered by the user is sent to the server via the device.

[1269] The server uses a generative AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. This AI model suggests optimal furniture placement and interior design based on historical and trend data. During the generation process, the server generates prompt messages based on the analyzed layout information and user input, and sends these prompt messages to the generative AI model.

[1270] As a concrete example, if a user requests a "resort hotel style" room design and sets a budget of 50,000 yen, the following prompt message will be generated: "I would like a resort hotel style room design. My budget is 50,000 yen. I have uploaded photos of the room. Based on this information, please generate three optimal design options."

[1271] The server generates optimal coordination suggestions based on data trained by a generation AI model and the latest trend data. Each generated coordination suggestion includes information on where to purchase the furniture used and its price. This allows users to easily create a room coordination that suits their preferences within their budget.

[1272] Ultimately, the device receives the coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links. The user can easily redecorate their room by selecting their favorite from the multiple coordination suggestions displayed and purchasing the furniture through the displayed links.

[1273] As described above, this system provides an effective means for users to easily coordinate their rooms and choose the optimal furniture within their budget.

[1274] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1275] Step 1:

[1276] Users upload room layouts and photos. Specifically, users use their devices to click the "Upload" button and select room layout and photo files. Then, they press the "Submit" button, and the selected files are sent from their device to the server. The input is the room layout and photos provided by the user, and the output is the image files sent to the server.

[1277] Step 2:

[1278] The server analyzes the received drawings and photographs. At this stage, the received image files are analyzed using image recognition libraries such as OpenCV or TensorFlow. The input is the image file sent to the server, and the output is the analyzed room layout information (room shape, location of walls, windows, doors, etc.). The server triggers the "start image recognition" process, analyzing the image pixel by pixel to extract room features.

[1279] Step 3:

[1280] The user inputs their preferred style and budget. Specifically, the user enters a style such as "resort hotel style" and a budget such as "50,000 yen" into a form provided on the device's interface. The form used for this is built with HTML or React. The input is the user's preferred style and budget, and the output is the style and budget information sent from the device to the server.

[1281] Step 4:

[1282] The server uses a generating AI model to create multiple coordination proposals based on the user's preferred taste, budget information, and analyzed floor plan information. The input is the user's taste information, budget information, and floor plan information, and the output is multiple coordination proposals generated by the generating AI model. The server combines the analysis results and the user's wishes to generate a prompt message and input it into the AI ​​model. For example, "I would like a resort hotel-style room coordination. My budget is 50,000 yen. I have uploaded photos of the room. Based on this information, please generate three optimal coordination proposals."

[1283] Step 5:

[1284] The server adds information about the source and price of the furniture used in the generated coordination proposals. The server retrieves source and price information from the database and integrates it into each coordination proposal. The input is the generated coordination proposal, and the output is the coordination proposal with added source and price information. Here, the server queries for specific store information and online shop links and integrates them into the proposal content.

[1285] Step 6:

[1286] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The input is the coordination suggestion information sent from the server, and the output is a detailed furniture layout diagram, purchase location, price information, and online purchase links displayed to the user. This information is displayed to the user on the device using HTML or React.

[1287] Step 7:

[1288] The user selects a displayed coordination plan and purchases the furniture. Specifically, the user selects a preferred coordination plan on their device and clicks the displayed online purchase link to complete the purchase process. The input is the user's selection of coordination plans and purchase process, and the output is the selection history and purchase information sent from the device to the server. The server stores this information in a database and uses it to optimize future suggestions.

[1289] (Application Example 1)

[1290] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1291] In modern life, interior design is an important element for many people, but there is a lack of easy and efficient ways to do it. Furthermore, the process of users selecting interior items that suit their taste and purchasing them within their budget is complex and cumbersome. Moreover, there is currently no system utilizing appropriate AI models to provide design suggestions, and these problems need to be addressed.

[1292] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1293] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using an AI model based on the input information and the layout, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, means for the user to select a displayed coordination proposal and purchase the furniture and update the server accordingly, means for visually displaying the interior design proposals generated by the AI ​​model via a smartphone app to support the purchase, means for displaying suggested content based on the user's preferences, and means for checking the where to purchase and the price of the furniture included in the interior design proposals online and performing the purchase procedure directly via the link. This makes it possible for users to easily check, select, and purchase interior coordination that suits their preferences and budget.

[1294] A "user" is an individual or corporation that uses this system to create room interior design plans or purchase furniture.

[1295] "Room floor plans and photos" refer to data containing visual information that users upload to the system to identify the layout and characteristics of a room.

[1296] A "server" is a central processing unit that analyzes drawings and photos received from users, generates coordination proposals, and also provides information on furniture suppliers and pricing.

[1297] "Analysis" refers to the process by which a server recognizes the layout and interior features of a room based on drawings and photographs.

[1298] "Preferred style and budget" refers to the interior style and amount of money that users can spend, which serve as the basis for generating room coordination suggestions.

[1299] An "AI model" refers to an artificial intelligence algorithm or computational model that uses input information and floor plan information to determine the optimal interior layout and furniture selection.

[1300] A "coordination plan" is a proposal generated by an AI model that shows the arrangement and selection of interior furnishings for a specific room.

[1301] "Purchase location and price information" refers to information about where the furniture included in the coordination plan can be purchased and how much it costs.

[1302] An "online purchase link" is a direct reference URL to a website or online store where users can immediately purchase the suggested furniture.

[1303] A "smartphone app" is application software that users can install to check room interior design ideas or purchase furniture.

[1304] "Selection history" refers to the record data of the coordination ideas and furniture that the user has selected and purchased so far.

[1305] "Optimization" refers to the process of adjusting the next suggestion to better match the user's needs, based on their preferences and past selection history.

[1306] This invention is a system in which a user uploads floor plans or photos of a room, and an AI model is used to generate room coordination suggestions based on that information. Specifically, the invention can be implemented through the following steps.

[1307] Hardware and software to be used

[1308] This system is implemented using the following hardware and software.

[1309] Smartphone: A device for applications that the user installs.

[1310] Server: Central processing unit that performs image analysis and inference processing using AI models.

[1311] OpenCV: A library for servers to perform image processing.

[1312] TensorFlow: A machine learning library for performing inference processing on AI models.

[1313] Database module: Manages furniture purchase information and pricing.

[1314] Program processing

[1315] The processes performed at each step of this system are as follows:

[1316] 1. Users upload room floor plans or photos:

[1317] Users upload room layouts and photos using a smartphone app. The uploaded layouts and photos are then sent from the smartphone to the server.

[1318] 2. The server analyzes the drawings and photos:

[1319] The server uses the OpenCV library to analyze the received drawings and photographs. The analysis automatically recognizes the room layout, extracting information such as the room's shape and the locations of walls, windows, and doors.

[1320] 3. The user enters their preferred taste and budget:

[1321] Users enter their preferred style (e.g., "resort hotel style") and budget on a smartphone app. The information entered by the user is sent to the server via the smartphone.

[1322] 4. The server generates a coordination proposal:

[1323] The server uses an AI model powered by TensorFlow to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model then suggests the optimal furniture placement and interior design based on past data and trends.

[1324] 5. The server adds the supplier and price information:

[1325] The server retrieves and adds information about the purchase locations and prices of the furniture used in the generated coordination plan from a database module. The process then integrates specific store information and online purchase links into the coordination plan.

[1326] 6. Your smartphone will display outfit suggestions:

[1327] The smartphone app receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[1328] 7. The user selects and purchases their favorite outfit:

[1329] Users select their favorite from several displayed coordination options and purchase the furniture via a link displayed through the smartphone app. The purchase information of the selected furniture is updated on the server and used to optimize future suggestions.

[1330] Specific example

[1331] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by information on where to buy it, its price, and an online purchase link. The user can then select their favorite room design and easily redecorate their room by purchasing the furniture via the link displayed on the smartphone app.

[1332] Example of a prompt

[1333] The following input prompts can be set for the generative AI model.

[1334] Prompt: Please provide images and floor plan information for the room. Then, generate the best interior design proposal to match the specified budget and style.

[1335] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1336] Step 1:

[1337] Users upload floor plans and photos of their rooms. Users use their smartphones to take photos and upload them to the server via the application. The input here is the floor plan and photo data sent from the user's device, while the output is the raw image data received on the server side.

[1338] Step 2:

[1339] The server analyzes drawings and photographs to recognize the room layout. The server uses OpenCV to extract structural information about the room within the image and automatically identifies the layout. The input here is the image data received in step 1, and the output is layout information such as the shape of the room and the positions of walls, windows, and doors. Specifically, the server resizes the image and extracts feature points to recognize the room layout.

[1340] Step 3:

[1341] The user enters their preferred taste and budget. The user sets their taste (e.g., "modern" or "classic") and budget within the application and sends this information to the server. The input here is the taste and budget information entered by the user, while the output is the preference and budget data received on the server side.

[1342] Step 4:

[1343] The server generates multiple coordination options using an AI model. Based on the received taste and budget information, and the analyzed floor plan information, the server creates coordination options from the AI ​​model using TensorFlow. The input here is the output data from steps 2 and 3, and the output is the multiple coordination options that have been generated. Specifically, the input data is passed to the AI ​​model, which calculates the optimal furniture placement and interior design.

[1344] Step 5:

[1345] The server adds information about the source and price of the furniture used in the coordination proposal. The server queries the database module for source and price information for the generated coordination proposal and adds the source information and price to each coordination. The input here is the coordination proposal generated in step 4, and the output is data with source and price information for the furniture associated with each coordination added.

[1346] Step 6:

[1347] The device displays coordination suggestions, purchase locations, price information, and online purchase links. The smartphone app visually displays coordination suggestions based on the information received from the server. The input here is the data generated in step 5, and the output is a display of coordination suggestions that the user can view. Specifically, the app screen displays a furniture layout diagram and purchase links for each item.

[1348] Step 7:

[1349] The user selects a displayed coordination suggestion and purchases the furniture. The user chooses their favorite from several displayed coordination suggestions and clicks the online purchase link via the smartphone app to purchase the furniture. The input here is the coordination suggestion displayed in step 6, and the output is information about the purchased items added to the shopping cart. Specifically, the user clicks the displayed link and completes the purchase procedure and payment. The purchase information of the selected furniture is updated on the server and used to optimize future suggestions.

[1350] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1351] This invention is a system in which a user uploads floor plans or photos of a room, and based on that, an AI model and emotion engine are used to generate room coordination suggestions. The system proposes multiple coordination suggestions that match the user's preferred style and budget, and also provides information on where to purchase the suggested furniture and their prices. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, the system optimizes the coordination suggestions according to the user's mood and past emotional data. The system's program processing is described below in natural language.

[1352] 1. The user uploads a floor plan or photos of the room.

[1353] Users upload room layouts and photos to the system using their devices. The uploaded layouts and photos are then sent from the device to the server.

[1354] 2. The server analyzes the drawings and photographs.

[1355] The server analyzes received drawings and photographs to automatically recognize the room layout. The recognized layout information includes the room's shape, the location of walls, windows, and doors. This analysis uses image recognition algorithms to extract the accurate layout.

[1356] 3. The user enters their preferred taste and budget.

[1357] The user accesses a form on their device to enter their preferred style (e.g., "resort hotel style") and budget. The information entered by the user is sent to the server via the device.

[1358] 4. The server generates a coordination proposal.

[1359] The server uses an AI model to generate multiple interior design options based on the received preferences, budget information, and analyzed room layout. The AI ​​model suggests optimal furniture placement and interior design based on past data and trends.

[1360] 5. The emotion engine recognizes the user's emotions.

[1361] The emotion engine analyzes the user's facial expressions and voice tone through the camera and microphone built into the device to recognize their current emotional state. This allows the server to provide real-time suggestions tailored to the user's mood.

[1362] 6. The server adds information about the supplier and price.

[1363] The server retrieves information on the source and price of the furniture used in the generated coordination plan from its database. This includes retrieving and integrating specific store information and online purchase links.

[1364] 7. The device displays outfit suggestions.

[1365] The device receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The displayed coordination suggestion includes specific furniture placement, purchase locations, price information, and online purchase links.

[1366] 8. The user selects and purchases their favorite outfit.

[1367] Users select their preferred outfit from several displayed options. In addition, an emotion engine evaluates the user's emotional response to their selection and adjusts the suggestions as needed. Selected furniture can be easily purchased by clicking the displayed online purchase link. The user's selection history and purchase information are sent from the device to the server and used to optimize future suggestions.

[1368] Specific example

[1369] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. Then, an emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, prioritizing and displaying the room design option that best suits their current mood. The user can easily redecorate their room by selecting a specific room design option and purchasing the furniture through the displayed link.

[1370] This system allows users to easily decorate their rooms to their liking, and through real-time suggestion optimization powered by an emotion engine, they can receive optimal suggestions that match their mood. Furthermore, the server's AI model constantly learns the latest trends and data, ensuring that the suggested decorating ideas are of high quality and relevant to the times.

[1371] The following describes the processing flow.

[1372] Step 1:

[1373] The user uses their device to select room floor plans or photo files and clicks the upload button.

[1374] Step 2:

[1375] The terminal sends the selected drawing or photo file to the server.

[1376] Step 3:

[1377] The server receives drawing and photo files sent from the terminal.

[1378] Step 4:

[1379] The server uses image analysis algorithms to recognize the room layout from the received drawings or photographs. This includes the process of extracting the locations of walls, windows, doors, and furniture.

[1380] Step 5:

[1381] The user accesses a form on their device interface to enter their preferred style (e.g., "resort hotel style") and budget.

[1382] Step 6:

[1383] The user enters the required information and clicks the submit button.

[1384] Step 7:

[1385] The device sends user-entered preferences and budget information to the server.

[1386] Step 8:

[1387] The server receives information about the user's preferred tastes and budget.

[1388] Step 9:

[1389] Based on floor plan information recognized by the server, as well as the user's preferred style and budget, an AI model is used to generate multiple design options.

[1390] Step 10:

[1391] For each generated design proposal, the server retrieves information on where to purchase the furniture and its price from the database. This includes obtaining specific store information and online purchase links.

[1392] Step 11:

[1393] The server retrieves the supplier and price information and adds it to each suggested outfit. If online purchase is possible, a link to that purchase is also included.

[1394] Step 12:

[1395] The server sends the outfit suggestions, purchase locations, pricing information, and online purchase links to the device.

[1396] Step 13:

[1397] The terminal receives coordination proposal information sent from the server and displays it to the user in a visually easy-to-understand format.

[1398] Step 14:

[1399] The user selects their favorite outfit from the displayed outfit suggestions.

[1400] Step 15:

[1401] The emotion engine analyzes the user's facial expressions and voice tone through the device's camera and microphone to recognize their current emotional state.

[1402] Step 16:

[1403] The server prioritizes displaying outfit suggestions that best suit the user's current mood, based on their emotional state.

[1404] Step 17:

[1405] The user clicks the online purchase link for the coordination plan they ultimately selected and buys the furniture online.

[1406] Step 18:

[1407] The device sends the user's selections and purchase history to a server, which is then recorded to optimize future suggestions.

[1408] (Example 2)

[1409] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1410] Conventional room coordination suggestion systems failed to consider the user's emotions and moods, making it difficult to increase user satisfaction. Furthermore, information on furniture suppliers and pricing based on the suggested coordination plans was scarce, making it difficult for users to make consistent purchase decisions without hassle. Additionally, there was a challenge in generating and presenting coordination plans that matched the user's preferences and budget in real time.

[1411] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1412] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for the server to generate multiple coordination proposals using an AI model based on the input information and the layout, means for an emotion engine to recognize the user's emotions and optimize the proposals based on that information, means for the server to add information on where to purchase and the price of the furniture used in the coordination proposals, means for the terminal to display the coordination proposals, where to purchase, price information, and online purchase links, and means for the user to select a displayed coordination proposal, purchase the furniture, and update the server accordingly. This makes it possible to provide more personalized coordination proposals that take into account the user's emotions and mood, to consistently provide information on where to purchase and the price of the furniture, and to improve user satisfaction.

[1413] - A "user" is the entity that uses this system to coordinate a room.

[1414] A "device" refers to an electronic device used by a user, such as a smartphone or personal computer.

[1415] A "server" is the central processing unit of a system, a computer that receives, analyzes, generates, and transmits data.

[1416] A "drawing" refers to a design plan or floor plan that shows the layout of a room.

[1417] A "photograph" is a still image that captures the current state of the room.

[1418] "Analysis" refers to the process of extracting floor plans and room features from uploaded drawings and photographs.

[1419] "Preferred taste" refers to the type of interior style or design that the user desires.

[1420] "Budget" refers to the maximum amount of money a user is willing to spend on coordinating an outfit.

[1421] An "AI model" is artificial intelligence that learns from past data and trends and generates outfit suggestions based on user input.

[1422] An "emotion engine" is software that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

[1423] "Purchase location" refers to the stores or online shops where the suggested furniture can be purchased.

[1424] "Price information" refers to information regarding the price of the proposed furniture.

[1425] An "online purchase link" is a URL that allows users to purchase furniture via the internet.

[1426] A "coordination plan" refers to furniture arrangements and interior designs proposed based on the user's preferred style, budget, and room layout.

[1427] "Selection history" refers to information about the coordination ideas and furniture that the user has previously selected in the system.

[1428] "Suggestion optimization" is the process of improving suggestions based on the user's selection history and emotional state.

[1429] This invention relates to a system in which a user uploads floor plans or photos of a room, and based on that, an AI model and an emotion engine are used to generate room coordination suggestions. The system allows the user to input their preferred style and budget, and then proposes multiple coordination suggestions that fit the budget, also providing information on where to purchase the suggested furniture and their prices. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, the system optimizes the coordination suggestions according to the user's mood and past emotional data.

[1430] Hardware and software

[1431] This system is implemented using the following hardware and software:

[1432] Server: This is a central processing unit that receives, analyzes, generates, and transmits data. Specifically, it uses cloud computing services such as Amazon Web Services (AWS) or Microsoft Azure.

[1433] Device: A smartphone or computer used by the user. These devices provide the user interface and allow for data uploading and display.

[1434] AI Model: Generative AI models such as Generative Adversarial Networks (GANs) are used to generate coordination proposals.

[1435] Emotion Engine: Utilizes Amazon Rekognition and Microsoft Azure's Emotion API to analyze the user's facial expressions and voice tone, and recognize their emotional state.

[1436] Image recognition algorithm: Using OpenCV and TensorFlow, analyze room layouts from uploaded drawings and photos.

[1437] Data processing and data calculation

[1438] 1. Uploading and receiving data:

[1439] Users upload room layouts and photos from their devices. This data is sent to the server via the internet.

[1440] 2. Image analysis:

[1441] The server stores the received drawings and photos and uses image recognition algorithms to analyze the room layout. This automatically recognizes the placement of walls, doors, and windows.

[1442] 3. Enter user information:

[1443] The user enters their preferred taste and budget into a form on their device. This information is then sent from the device to the server.

[1444] 4. Generating coordination proposals:

[1445] The server uses an AI model to generate multiple coordination options based on user input and analyzed floor plan data.

[1446] 5. Emotion recognition:

[1447] The emotion engine analyzes the user's facial expressions and voice tone through the device's camera and microphone, recognizing their emotional state in real time. This information is sent to a server and used to optimize suggestions.

[1448] 6. Add purchase information:

[1449] The server retrieves furniture supplier and price information from the database for each generated coordination plan and integrates it into the plan.

[1450] 7. Display of outfit suggestions:

[1451] The terminal receives coordination proposal information sent from the server and displays it to the user in a visually easy-to-understand format. The display includes furniture layout diagrams, purchase locations, price information, and online purchase links.

[1452] 8. Selection and Purchase:

[1453] Users can purchase furniture online based on their selected coordination plan. Purchase information is sent to the server and stored to optimize future suggestions.

[1454] Specific example

[1455] For example, suppose a user wants a "resort hotel style" room and sets a budget of 50,000 yen. When the user uploads a photo of their room, the server analyzes the layout and generates three different room design options based on that information and the user's preferences. Each option includes the placement of a specific sofa, rug, and table, and each piece of furniture is accompanied by store information, price, and an online purchase link. Then, an emotion engine analyzes the user's facial expressions and tone of voice through the device's camera and microphone, prioritizing and displaying the room design option that best suits their current mood. The user can easily redecorate their room by selecting a specific room design option and purchasing the furniture through the displayed link.

[1456] Example of a prompt

[1457] Based on the following room photos and floor plan information, please propose three resort-hotel-style living room design ideas within a budget of 50,000 yen. [Attach room photos and floor plan information]

[1458] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1459] Step 1:

[1460] The user uploads room plans and photos using their device. Specifically, the user opens a file selection dialog on their device and selects the room plans and photos. The input here is the room plans and photos, which are sent from the device to the server as an HTTP POST request. The output is the data of the plans and photos transferred to the server.

[1461] Step 2:

[1462] The server analyzes the drawings and photos it receives. Specifically, it temporarily stores the received files and extracts room layout information using an image recognition algorithm (e.g., OpenCV or TensorFlow). The input is the uploaded drawing or photo file, and the output is room layout information (location of walls, doors, windows, etc.).

[1463] Step 3:

[1464] The user enters their preferred style and budget through the terminal's interface. The user enters their style (e.g., "resort hotel style") and budget into the form and presses the "Submit" button. The input consists of style and budget information, which is sent from the terminal to the server. The output is the submitted style and budget data.

[1465] Step 4:

[1466] Based on the server's received preferences, budget information, and analyzed floor plan data, an AI model is used to generate multiple coordination proposals. Specifically, generative AI models such as Generative Adversarial Networks (GANs) are used. The inputs are preferences, budget information, and floor plan data, and the output is multiple coordination proposals.

[1467] Step 5:

[1468] The emotion engine analyzes the user's facial expressions and voice tone through the device's built-in camera and microphone, recognizing their emotional state in real time. The input is data on the user's facial expressions and voice tone, which the emotion engine analyzes. The output is data on the user's current emotional state.

[1469] Step 6:

[1470] The server receives emotional state data from the emotion engine and optimizes the suggested outfit combinations. Specifically, this data is used to provide suggestions that are tailored to the user's emotional state in real time. The input is emotional state data, and the output is an outfit combination optimized according to that emotion.

[1471] Step 7:

[1472] The server retrieves furniture supplier and price information from a database for each generated coordination proposal and integrates it into the proposal. Specifically, it issues queries to the APIs of various online stores to obtain appropriate furniture supplier and price information. The input is data on the coordination proposal, supplier, and price information, and the output is the coordination proposal with the supplier and price information added.

[1473] Step 8:

[1474] The terminal receives coordination suggestion information sent from the server and displays it to the user in a visually easy-to-understand format. The display includes furniture layout diagrams, purchase locations, price information, and online purchase links. The input is the coordination suggestion information sent from the server, and the output is the visual information displayed on the terminal.

[1475] Step 9:

[1476] The user selects their preferred furniture from several displayed coordination options and purchases it online. The selection is made through the terminal's interface, and clicking the purchase link initiates the online purchase process. Input is the user's selection information, and output is data confirming the completion of the purchase. The server receives this selection history and stores it to optimize future suggestions.

[1477] The above describes the specific processing flow of this system and the detailed operation at each step.

[1478] (Application Example 2)

[1479] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1480] In today's world, interior design has become incredibly diverse, making it a challenging task to automatically suggest the optimal design based on user preferences and budgets. Furthermore, real-time optimization based on user emotions and moods has not been implemented, and systems that reduce the hassle of purchasing are still lacking. Additionally, virtual environments where users can physically examine furniture online before purchasing are not adequately provided.

[1481] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1482] In this invention, the server includes means for the user to upload room plans or photos, means for the server to receive and analyze the uploaded plans or photos to recognize the room layout, means for the user to input their preferred style and budget and means for the server to receive this information, means for generating coordination proposals using an AI model, means for adding information on where to purchase and the prices of the furniture used in the coordination proposals, means for analyzing the user's emotional state and optimizing the coordination proposals based on those emotions, and means for checking and selecting furniture in a virtual environment. As a result, the user can receive suggestions for the best interior coordination according to their preferences and budget, and furthermore, by optimizing suggestions based on their emotional state, they can obtain coordination proposals that match their mood in real time and purchase furniture while checking it in a virtual environment.

[1483] A "user" refers to a person who uses this system to generate room coordination ideas.

[1484] "Room floor plans and photos" refers to the layout information of a room or images taken by the user.

[1485] A "server" refers to a computer system that receives information sent by users, analyzes it, generates coordination proposals, and provides purchase information.

[1486] "Floor plan" refers to information about the layout of a room, such as its shape and the location of walls, windows, and doors.

[1487] "Preferred taste" refers to the type of interior style that the user desires.

[1488] "Budget" refers to the amount of money a user is willing to pay for an outfit.

[1489] An "AI model" refers to an algorithm used to generate outfit suggestions based on past data and trends.

[1490] "Coordination proposal" refers to the interior design suggestions for a room generated by the server.

[1491] "Emotional state" refers to the user's current emotions, obtained by analyzing their facial expressions and tone of voice.

[1492] "Purchase source" refers to stores or online shops where you can buy the furniture used in the coordination.

[1493] "Price information" refers to the price of the furniture offered by the retailer.

[1494] An "online purchase link" refers to a URL that allows users to purchase the furniture used in the suggested interior design over the internet.

[1495] A "virtual environment" refers to a virtual space that resembles reality, created using computer graphics and other technologies.

[1496] "Emotional analysis means" refers to technology that analyzes a user's emotional state in real time through a camera or microphone.

[1497] This invention is a system that allows users to upload floor plans or photos of their rooms, generate room coordination suggestions using an AI model and emotion engine based on those plans, and then view and select furniture within a virtual environment.

[1498] Hardware and software to use

[1499] Hardware:

[1500] Terminal devices such as smartphones, smart glasses, and head-mounted displays

[1501] Camera and microphone (for the emotion engine)

[1502] software:

[1503] Image recognition algorithm (OpenCV)

[1504] Emotion recognition engine (Microsoft Azure Cognitive Services)

[1505] AI model (AWS SageMaker or Google Cloud AI)

[1506] Database (MySQL or Firestore)

[1507] System processing details

[1508] 1. Users upload room floor plans or photos:

[1509] Users upload room layouts and photos to the system using a terminal device. This uploaded data is then sent from the terminal to the server.

[1510] 2. The server analyzes the drawings and photographs:

[1511] The server uses OpenCV to analyze received drawings and photos, automatically recognizing the room layout. This layout information includes the shape of the room, and the locations of walls, windows, and doors.

[1512] 3. The user enters their preferred taste and budget:

[1513] The user accesses a form on their device to enter their preferred taste and budget. The entered information is then sent to the server via the device.

[1514] 4. The server generates a coordination proposal:

[1515] Based on the received taste and budget information, along with the analyzed floor plan information, the server uses an AI model (AWS SageMaker or Google Cloud AI) to generate multiple coordination proposals.

[1516] 5. The emotion engine recognizes the user's emotions:

[1517] Through the camera and microphone built into the device, the emotion engine (Microsoft Azure Cognitive Services) analyzes the user's facial expressions and tone of voice to recognize their current emotional state. This allows the server to provide real-time suggestions tailored to the user's mood.

[1518] 6. The server adds information about the supplier and price:

[1519] The server retrieves information on the source and price of the furniture used in the generated coordination plan from a database (MySQL or Firestore) and adds it to the plan.

[1520] 7. The device displays outfit suggestions:

[1521] The device receives coordination suggestions sent from the server and displays them to the user in a visually easy-to-understand format. The displayed coordination suggestions include furniture placement, purchase locations, price information, and online purchase links.

[1522] 8. View and select furniture within the virtual environment:

[1523] Users can use smart glasses or head-mounted displays to view and select furniture in a virtual environment. The selected and purchased information is updated on the server and used to improve future recommendations.

[1524] Adding specific examples

[1525] For example, suppose a user wants a "Nordic-style" interior and sets a budget of 30,000 yen. When the user uploads photos of their room, the server analyzes the layout, and based on that information and the user's preferences, an AI model generates three different interior design options. An emotion engine analyzes the user's mood and suggests the most suitable option. The user can use smart glasses to walk around the virtual store, view, select, and purchase furniture.

[1526] Example of a prompt

[1527] "Design a system where users input their desired interior style and budget, upload room plans and photos, and then use AI and an emotion engine to suggest the optimal furniture arrangement and design based on that information."

[1528] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1529] Step 1:

[1530] Users upload floor plans and photos of their rooms.

[1531] Input: Room floor plan or photo

[1532] Output: Uploaded drawing or photo data

[1533] Specific operation: Users use devices such as smartphones or tablets to select room layouts and photo files and upload them to the system. This data is sent to the server via the internet.

[1534] Step 2:

[1535] The server receives drawings and photos, analyzes them, and recognizes the floor plan.

[1536] Input: Uploaded drawing or photo data

[1537] Output: Analyzed floor plan information (room shape, wall, window, door locations, etc.)

[1538] Specific operation: The server receives the transmitted image data and analyzes the drawings and photographs using OpenCV. The analysis algorithm recognizes each element of the room (walls, windows, doors, etc.) and structures this as floor plan information.

[1539] Step 3:

[1540] The user enters their preferred taste and budget.

[1541] Input: Preferred style (e.g., "Nordic style"), budget

[1542] Output: Input taste and budget data

[1543] Specific operation: The user accesses a form on their device screen and enters their preferred interior style and estimated budget. This information is sent to the server via the internet.

[1544] Step 4:

[1545] The server uses an AI model to generate multiple coordination options based on the input information and floor plan.

[1546] Input: Taste, budget, floor plan information

[1547] Output: Multiple coordination options

[1548] Specific operation: The server combines the received taste, budget, and floor plan information and uses an AI model to generate the optimal coordination plan. The AI ​​model uses past datasets and trend information to suggest furniture placement and design.

[1549] Step 5:

[1550] The emotion engine recognizes the user's emotions.

[1551] Input: User's facial expressions and tone of voice

[1552] Output: Analyzed user emotional state

[1553] Specific operation: The emotion recognition engine analyzes the user's facial expressions and voice tone through the camera and microphone on the user's device. The results of this analysis are sent to the server.

[1554] Step 6:

[1555] The server optimizes outfit suggestions based on the user's emotional state and adds information about where to purchase the items and their prices.

[1556] Input: Multiple outfit ideas, user's emotional state

[1557] Output: Optimized outfit suggestions, with purchase location and pricing information.

[1558] Specific operation: The server optimizes outfit suggestions based on the received emotional state. Next, it retrieves and adds supplier and price information from the database for each optimized suggestion.

[1559] Step 7:

[1560] The device displays coordination suggestions, allowing users to view furniture within a virtual environment.

[1561] Input: Optimized outfit suggestions, purchase locations, and price information

[1562] Output: User-confirmable outfit suggestions, purchase locations, and price information.

[1563] Specific operation: The device receives data sent from the server and displays it visually to the user. This display uses 3D models or AR technology, allowing users to view furniture in a virtual environment.

[1564] Step 8:

[1565] The user selects a displayed coordination suggestion and purchases the furniture.

[1566] Input: Selection information for coordination options, purchase intent.

[1567] Output: Purchase completion status, information on selected furniture.

[1568] Specific operation: The user selects their preferred coordination plan on their device and presses the purchase button to actually order the furniture via an online purchase link. The selection and purchase history is sent to the server and used for future suggestions.

[1569] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1570] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1571] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1572] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1573] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1574] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1575] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1576] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1577] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1578] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1579] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1580] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1581] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1582] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1583] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1584] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1585] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1586] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1587] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1588] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1589] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1590] The following is further disclosed regarding the embodiments described above.

[1591] (Claim 1)

[1592] A means for users to upload room plans and photos,

[1593] A server receives and analyzes uploaded drawings and photographs to recognize the room layout,

[1594] A means by which the user inputs their preferred taste and budget, and a means by which the server receives that information.

[1595] A server provides a means for generating multiple coordination proposals using an AI model based on input information and floor plans,

[1596] The server provides a means to add information on the source and price of the furniture used in the coordination plan,

[1597] The device provides a means for displaying outfit suggestions, purchase locations, price information, and online purchase links.

[1598] A means by which a user selects a displayed coordination plan and purchases furniture, and a means of updating that plan on the server.

[1599] A system that includes this.

[1600] (Claim 2)

[1601] The system according to claim 1, comprising means for generating outfit suggestions from a specific store designated by the user.

[1602] (Claim 3)

[1603] The system according to claim 1, comprising means for optimizing the next suggestion based on the user's selection history.

[1604] "Example 1"

[1605] (Claim 1)

[1606] A means for users to upload room plans and photos,

[1607] A server receives and analyzes uploaded drawings and photographs to recognize the room layout,

[1608] A means by which the user inputs their preferred taste and budget, and a means by which the server receives that information.

[1609] A server generates multiple coordination proposals using an AI model based on input information and floor plans,

[1610] The server provides a means to add information on the source and price of the furniture used in the coordination plan,

[1611] The device provides a means for displaying outfit suggestions, purchase locations, price information, and online purchase links.

[1612] A means for the user to select a displayed coordination plan and purchase furniture, and a means for updating it on the server.

[1613] A means for generating prompt text based on floor plan information analyzed by the server and user input information, and sending it to a generation AI model,

[1614] A server-generated AI model generates optimal coordination suggestions based on trained data and trend data.

[1615] A system that includes this.

[1616] (Claim 2)

[1617] The system according to claim 1, comprising means for generating outfit suggestions from a specific store designated by the user.

[1618] (Claim 3)

[1619] The system according to claim 1, comprising means for optimizing the next suggestion based on the user's selection history.

[1620] "Application Example 1"

[1621] (Claim 1)

[1622] A means for users to upload room plans and photos,

[1623] A server receives and analyzes uploaded drawings and photographs to recognize the room layout,

[1624] A means by which the user inputs their preferred taste and budget, and a means by which the server receives that information.

[1625] A server provides a means for generating multiple coordination proposals using an AI model based on input information and floor plans,

[1626] The server provides a means to add information on the source and price of the furniture used in the coordination plan,

[1627] The device provides a means for displaying outfit suggestions, purchase locations, price information, and online purchase links.

[1628] A means for the user to select a displayed coordination plan and purchase furniture, and a means for updating it on the server.

[1629] An AI model generates interior design proposals which are then visually displayed via a smartphone app to support the purchase process.

[1630] A means of displaying suggestions based on user preferences,

[1631] A method to check online where to purchase furniture included in the interior design plan and at what price, and to purchase it directly via a link.

[1632] A system that includes this.

[1633] (Claim 2)

[1634] The system according to claim 1, comprising means for generating outfit suggestions from a specific store designated by the user.

[1635] (Claim 3)

[1636] The system according to claim 1, comprising means for optimizing the next suggestion based on the user's selection history.

[1637] "Example 2 of combining an emotion engine"

[1638] (Claim 1)

[1639] A means for users to upload room plans and photos,

[1640] A server receives and analyzes uploaded drawings and photographs to recognize the room layout,

[1641] A means by which the user inputs their preferred taste and budget, and a means by which the server receives that information.

[1642] A server provides a means for generating multiple coordination proposals using an AI model based on input information and floor plans,

[1643] An emotion engine recognizes the user's emotions and optimizes suggestions based on that information,

[1644] The server provides a means to add information on the source and price of the furniture used in the coordination plan,

[1645] The device provides a means for displaying outfit suggestions, purchase locations, price information, and online purchase links.

[1646] A means by which a user selects a displayed coordination plan and purchases furniture, and a means of updating that plan on the server.

[1647] A system that includes this.

[1648] (Claim 2)

[1649] The system according to claim 1, comprising means for generating outfit suggestions from a specific store designated by the user.

[1650] (Claim 3)

[1651] The system according to claim 1, comprising means for optimizing the next suggestion based on the user's selection history.

[1652] "Application example 2 when combining with an emotional engine"

[1653] (Claim 1)

[1654] A means for users to upload room plans and photos,

[1655] A server receives and analyzes uploaded drawings and photographs to recognize the room layout,

[1656] A means by which the user inputs their preferred taste and budget, and a means by which the server receives that information.

[1657] A server provides a means for generating multiple coordination proposals using an AI model based on input information and floor plans,

[1658] The server provides a means to add information on the source and price of the furniture used in the coordination plan,

[1659] The device provides a means for displaying outfit suggestions, purchase locations, price information, and online purchase links.

[1660] A means for the user to select a displayed coordination plan and purchase furniture, and a means for updating it on the server.

[1661] A means for analyzing the user's emotional state and a means for optimizing coordination suggestions based on those emotions,

[1662] A system that includes means for viewing and selecting furniture within a virtual environment.

[1663] (Claim 2)

[1664] The system according to claim 1, comprising means for generating outfit suggestions from a specific store designated by the user.

[1665] (Claim 3)

[1666] The system accor...

Claims

1. A means for users to upload room plans and photos, A server receives and analyzes uploaded drawings and photographs to recognize the room layout, A means by which the user inputs their preferred taste and budget, and a means by which the server receives that information. A server provides a means for generating multiple coordination proposals using an AI model based on input information and floor plans, The server provides a means to add information on the source and price of the furniture used in the coordination plan, The device provides a means for displaying outfit suggestions, purchase locations, price information, and online purchase links. A means by which a user selects a displayed coordination plan and purchases furniture, and a means of updating that plan on the server. A system that includes this.

2. The system according to claim 1, comprising means for generating outfit suggestions from a specific store designated by the user.

3. The system according to claim 1, comprising means for optimizing the next suggestion based on the user's selection history.

Images