System

Abstract

A system is provided.SOLUTION: Input means for inputting a photograph of a room, size information of the room, a necessary furniture and appliance list, owned furniture and appliance information, and a favorite style from a user; transmission means for transmitting data from the input means to a server; analysis means for receiving data from the transmission means and performing image analysis and text data analysis; The system includes a proposal means for generating a furniture / household appliance list to be proposed, a link generation means for generating a link to a purchase page of the furniture / household appliance proposed by the proposal means, and a display means for transmitting and displaying data generated by the link generation means to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When moving into a new home, it is difficult to plan the ideal interior design and layout of home appliances, often resulting in purchases that don't quite fit. This problem arises from a lack of means to select interior design and home appliances that suit the user's preferences, optimize the room size and layout, and visually confirm these. Conventional technology has not fully met these user needs, and an efficient and effective room coordination method is needed. [Means for solving the problem]

[0005] The present invention provides a system including an input means for a user to input photos of the room, room size information, a list of necessary furniture and home appliances, information on furniture and home appliances on hand, and preferred tastes; a transmission means for transmitting data from the input means to a server; an analysis means for receiving the data from the transmission means and performing image analysis and text data analysis; a generation means for generating a room layout image using a generation AI based on the results of the analysis means; a proposal means for generating a list of suggested furniture and home appliances based on the layout generated by the generation means; a link generation means for generating links to purchase pages for the furniture and home appliances proposed by the proposal means; and a display means for transmitting and displaying the data generated by the link generation means to the user.

[0006] This system not only allows users to efficiently and effectively select and arrange furniture and appliances that suit their room and preferences, but also provides links to purchase each item, making it easy for users to create their ideal room.

[0007] The "input means" is a means by which the user inputs a photo of the room, information about the size of the room, a list of necessary furniture and appliances, information about the furniture and appliances that the user already has, and their preferred tastes.

[0008] The "transmission means" is a means for transmitting data from the input means to the server.

[0009] The "analysis means" is a means for receiving data from the transmission means and performing image analysis and text data analysis.

[0010] The "generation means" is a means for generating a room layout image using a generation AI based on the results of the analysis means.

[0011] The "proposing means" is a means for generating a list of furniture and home appliances to be proposed based on the layout generated by the generating means.

[0012] The "link generating means" is a means for generating a link to a purchase page for the furniture / home appliances proposed by the suggesting means.

[0013] The "display means" is a means for transmitting the data generated by the link generation means to the user and displaying it. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room when moving into a new home. This system uses a generative AI to generate a room layout image based on user input and suggests specific interior decor and home appliances. The processing flow of the system's program and its specific operation are explained below.

[0036] System Overview

[0037] This system consists of a terminal and a server. The terminal sends the user's input information to the server and displays the final result to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list.

[0038] Program processing flow and operation

[0039] 1. User Input

[0040] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they currently have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style (simple modern).

[0041] 2. Data Transmission

[0042] The terminal transmits the collected user input data to the server.

[0043] 3. Data Receipt and Storage

[0044] The server receives the data sent from the device and stores it in a database for analysis.

[0045] 4. Image Analysis

[0046] The server retrieves a photo of the room from the database and performs image analysis, using image recognition algorithms to identify the room's layout and the location and dimensions of existing furniture, for example, using edge detection and object recognition algorithms to identify the contours of the room and the location of furniture.

[0047] 5. Text Data Analysis

[0048] The server analyzes the room size information, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, thereby understanding the size of the room and the required interior specifications.

[0049] 6. Taste Analysis

[0050] The server performs an analysis to identify suitable furniture and appliances based on the user's preferred style (e.g., simple modern).

[0051] 7. Data integration and input information organization

[0052] The server integrates and organizes the results of image analysis, text data analysis, and taste analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[0053] 8. Layout Generation Using Generative AI

[0054] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then uses the user's input information and analysis results to automatically arrange the layout and interior.

[0055] 9. Generate a list of interior, furniture, and home appliances

[0056] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a simple, modern TV stand or carpet to the list.

[0057] 10. Generate shopping page links

[0058] The server generates a link to a purchasing page for each suggested piece of furniture or appliance, possibly using an online shopping API.

[0059] 11. Data Transmission

[0060] The server sends the device an image of the final room layout, a list of suggested furniture and appliances, and a link to purchase each item.

[0061] 12. Displaying Data

[0062] The device displays to the user an image of the final room layout, a list of suggested furniture and appliances, and a link to purchase.

[0063] 13. User Choices and Purchases

[0064] Users can check the information displayed on their device, select the furniture or home appliances they like from the suggested items, and purchase them using the provided purchase link.

[0065] This concludes the description of the embodiment of the invention. This system not only allows users to efficiently and effectively select and arrange furniture and home appliances that suit their room and preferences, but also provides links to purchase each item at the same time. This allows users to easily create their ideal room.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they already have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), that they need a TV stand and carpet, that they already have a sofa and table, and that their preferred style is simple and modern.

[0069] Step 2:

[0070] The terminal collectively transmits the data input by the user to the server.

[0071] Step 3:

[0072] The server receives the data sent from the device and stores it in a database for analysis.

[0073] Step 4:

[0074] The server retrieves the room's photo data from the database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. For example, edge detection algorithms and object recognition techniques are used to detect the contours of the room and the location of furniture.

[0075] Step 5:

[0076] The server analyzes the text data provided by the user, including the room size, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, to understand the room size and required interior specifications.

[0077] Step 6:

[0078] The server performs analysis to identify suitable furniture and appliances based on the user's preferred style (e.g., simple modern).

[0079] Step 7:

[0080] The server integrates and organizes the results of image analysis, text data analysis, and taste analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[0081] Step 8:

[0082] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then automatically arranges furniture and appliances based on the user's input data and analysis results to create the optimal layout.

[0083] Step 9:

[0084] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the AI. For example, it adds a TV stand or carpet with a simple, modern design to the list.

[0085] Step 10:

[0086] The server generates a link to the online shopping site for each proposed piece of furniture or appliance, taking into account the size and placement information of each item.

[0087] Step 11:

[0088] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[0089] Step 12:

[0090] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[0091] Step 13:

[0092] Users can select the furniture or appliances they like from the suggested list displayed on their device and complete the purchase using the provided link. At this time, they can also check the specific interior layout while making their selection.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] In conventional interior and home appliance purchasing support systems, it has been difficult for users to effectively plan the layout, interior, and placement of home appliances in a room. In particular, there was a lack of a way to generate an optimal layout while taking into account the user's preferences, existing furniture, and room size. In addition, generating purchase links had to be done manually, which was a cumbersome and time-consuming task for users. These challenges made it difficult to efficiently create the ideal room.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes input means for allowing a user to input a photo of the room, room dimension information, a list of necessary fixtures, information about fixtures on hand, and a preferred design style, transmission means for transmitting data from the input means to the server, analysis means for receiving the data from the transmission means and performing image analysis and text data analysis, generation means for generating a room layout image using a generative AI model based on the results of the analysis means, proposal means for generating a proposed fixture list based on the layout generated by the generation means, link generation means for generating links to purchasing pages for the fixtures proposed by the proposal means, and display means for transmitting and displaying the data generated by the link generation means to the user. This enables users to efficiently design a room with an ideal layout based on their preferences and easily purchase the necessary fixtures.

[0098] A "user" is an individual or organization that uses the system to receive support for room layout, interior design, and purchasing home appliances.

[0099] A "terminal" is a device used by a user to provide input information to the system, and includes a PC, smartphone, tablet, etc.

[0100] The "server" is a central processing unit that receives data sent from users, analyzes it, and generates layout images and proposal lists.

[0101] The "input means" is an interface that allows the user to input a photo of the room, dimension information, a list of necessary equipment, information on equipment that is on hand, and a preferred design style.

[0102] The "transmission means" is a means for transmitting the user's input data from the terminal to the server.

[0103] The "analysis means" is a means for analyzing received data in the server and performing image analysis and text data analysis.

[0104] The "generation means" is a means for generating a room layout image using a generative AI model based on the results of the analysis means.

[0105] The "proposing means" is a means for generating an equipment list to be proposed to the user based on the layout generated by the generating means.

[0106] The "link generating means" is a means for generating a link to a purchasing page for the equipment suggested by the suggesting means.

[0107] The "display means" is a means for transmitting the data generated by the link generation means to the user and displaying it.

[0108] "Image analysis" is a technology that processes photos of a room sent by a user and identifies the outline of the room and the location of furniture.

[0109] "Text data analysis" is a technology that analyzes the room dimension information, required equipment list, equipment information on hand, and preferred design style sent by the user.

[0110] A "generative AI model" is an artificial intelligence model that generates optimal room layout images based on user input information and analysis results.

[0111] A "prompt sentence" is an input sentence used to give instructions to a generative AI model.

[0112] "Fixtures" is a general term for decorative and functional items, including interior furnishings and home appliances.

[0113] "Design style" is a concept that refers to a particular theme or aesthetic of room decoration or interior that a user prefers.

[0114] This invention is a system that efficiently supports the purchase of interior decor and home appliances to create an ideal room when a user moves into a new residence. This system is composed of a terminal and a server. The terminal sends the user's input information to the server, and the server uses a generative AI model to generate a room layout image and a proposal list. The final data is then provided to the user.

[0115] Input Method

[0116] Users use their device to input a photo of their room, its dimensions, a list of necessary furnishings, information about the furnishings they already have, and their preferred design style. For example, a user can take a photo of their living room with their smartphone and upload it to the application. They can also input the room dimensions (5 meters long and 4 meters wide), select a TV stand and carpet as necessary items, input the sofa and table they already have, and set their preferred design style to simple modern.

[0117] Transmission method

[0118] The terminal sends the user's input data collected through the input means to the server. This transmission is securely performed using the HTTPS protocol. The input data is converted to JSON format, encrypted, and sent to the server.

[0119] Analysis means

[0120] The server receives data sent from the device and stores it in a database for analysis. For image analysis, the open-source image processing library "OpenCV" is used, using edge detection and object recognition algorithms to identify the outline of the room and the location of furniture. For text data analysis, the "NLTK" library is used to analyze room dimensions and a list of necessary equipment. For taste analysis, generative AI models such as "GPT-3" are used.

[0121] generation means

[0122] Based on the analysis results, the server uses a generative AI model to generate a room layout image. Specifically, it uses a generative AI model such as "DALL·E" and generates the optimal room layout image by inputting a text prompt. The following text is an example of a prompt that can be used:

[0123] Example prompt sentence:

[0124] User input details:

[0125] Room photo

[0126] Room size: 5 meters by 4 meters

[0127] Required furniture and appliances: TV stand and carpet

[0128] Existing items: Sofa and table

[0129] Preferred style: Simple Modern

[0130] Please generate an optimal room layout image and suggest a list of furniture and appliances.

[0131] Proposal means

[0132] The server then creates a list of suggested furnishings for the user based on the generated layout image. For example, it adds a TV stand and carpet suitable for a simple modern style to the list. This suggestion is based on the analyzed user preferences.

[0133] Link Generation Methods

[0134] For each proposed item, the server generates a link to an online shopping site to purchase the item. Specifically, it uses the API of a popular online marketplace to retrieve the product link.

[0135] Display means

[0136] The server sends the final room layout image, the proposed equipment list, and a purchase link for each item to the terminal, which then displays this information to the user. For example, the user can view the layout image and the proposed equipment list through the application interface and click the purchase link to go to a shopping site.

[0137] User Choice and Purchase

[0138] Users can select their favorite items based on the suggested information and complete the purchase process using the provided purchase link, allowing them to efficiently create their ideal room.

[0139] The system of the present invention allows users to simultaneously obtain layout suggestions that suit their room and preferences, a list of necessary fixtures, and links to purchase each item, making it easy for users to create their ideal room.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1:

[0142] The user uses the terminal to input a photo of the room, room dimensions, a list of necessary equipment, information on equipment they have on hand, and their preferred design style. The information entered by the user is entered into an input form using text boxes and selection lists. The entered data is stored in the system's internal memory.

[0143] Input: The user inputs a photo of the room, dimension information, a list of necessary equipment, information on equipment on hand, and design style through the terminal.

[0144] Output: Input data stored in internal memory.

[0145] Specific operation: The user takes a photo of their living room with their smartphone camera and uploads it to the application. They also input the room dimensions (5 meters long x 4 meters wide), select a TV stand and carpet as necessary items, input information about their existing sofa and table, and set their preferred design style to simple modern.

[0146] Step 2:

[0147] The terminal sends the data entered by the user to the server, where the transmission is encrypted via the HTTPS protocol, ensuring reliable communication.

[0148] Input: Input data stored in the internal memory.

[0149] Output: The input data sent to the server.

[0150] Specific behavior: When the user taps the "Submit" button, the entered data is converted to JSON format and sent to the server using the HTTPS protocol.

[0151] Step 3:

[0152] The server receives the data sent from the device, stores it in an analysis database, and uses it in subsequent analysis processes.

[0153] Input: Input data sent from the terminal.

[0154] Output: The input data stored in a database.

[0155] Specific operation: The server parses the received data in JSON format, extracts each item (photo, dimension information, equipment list, design style, etc.), and stores them in a database.

[0156] Step 4:

[0157] The server retrieves photos of the room from the database and performs image analysis using OpenCV, which uses edge detection and object recognition algorithms to identify the contours of the room and the location of furniture.

[0158] Input: A photo of a room stored in a database.

[0159] Output: Room outline and furniture position information (e.g. coordinate data).

[0160] Specific operation: Edges are detected using the OpenCV Canny algorithm, the shape of the furniture is identified using the Hough transform, and the type and location of the furniture are identified using an object recognition algorithm.

[0161] Step 5:

[0162] The server retrieves text data (dimension information, list of required equipment, information on equipment on hand, design style) from the database and performs text data analysis.

[0163] Input: Text data stored in a database.

[0164] Output: Analyzed dimensions, required equipment list, on-hand equipment information, design styles.

[0165] How it works: The server uses NLTK to parse the text data, extract dimensional information as numerical data, obtain the required equipment list and on-hand equipment information, and classify the relevant information based on the design style.

[0166] Step 6:

[0167] The server then integrates the text data and image analysis results and invokes a generative AI model to generate the optimal room layout. This process uses tools such as "DALL·E."

[0168] Input: Analyzed dimensions, required equipment list, equipment on hand, design style, image analysis results (room outline and furniture position information).

[0169] Output: Generated room layout image.

[0170] Specific operation: The server generates a prompt sentence and inputs it into the generative AI model along with the analysis data. The generative AI model then generates the optimal layout image based on this and sends it back to the server. Examples of prompt sentences include:

[0171] User input details:

[0172] Room photo

[0173] Room size: 5 meters by 4 meters

[0174] Required furniture and appliances: TV stand and carpet

[0175] Existing items: Sofa and table

[0176] Preferred style: Simple Modern

[0177] Please generate an optimal room layout image and suggest a list of furniture and appliances.

[0178] Step 7:

[0179] The server creates a list of furnishings to suggest to the user based on the generated layout image, and selects related interior and home appliances based on the analyzed design style.

[0180] Input: The generated room layout image.

[0181] Output: Proposed equipment list.

[0182] Specific behavior: The server adds related items to the list, such as TV stands and carpets that fit the simple modern style. This suggestion list is created based on the results of image and text analysis.

[0183] Step 8:

[0184] The server generates a link to an online shopping site for each proposed item, obtains the product link via API, and provides it to the user.

[0185] Input: Proposed equipment list.

[0186] Output: Purchase links for each fixture.

[0187] Specific operation: The server calls the API of the online shopping site, obtains the purchase link for each suggested item, and adds it to the list.

[0188] Step 9:

[0189] The server sends the final generated room layout image, the proposed equipment list, and a purchase link to the terminal.

[0190] Input: Generated room layout image, proposed fixture list, purchase link.

[0191] Output: The total data sent to the device.

[0192] What it does: The server converts the relevant data into a user-friendly HTML format and sends it to the device, where users can preview the interior and purchase items.

[0193] Step 10:

[0194] The terminal displays to the user an image of the final room layout, a list of suggested furnishings, and a link to purchase.

[0195] Input: Comprehensive data sent from the server.

[0196] Output: A room layout image shown to the user, a suggested furniture list, and a purchase link.

[0197] Specific operation: The user can view the layout image and suggested equipment list through the application on their device, and click on the provided link to go to the shopping site to purchase the items.

[0198] Through the above series of processing steps, the present invention provides a system that allows users to easily and efficiently realize their ideal room and purchase the necessary interior decorations and home appliances.

[0199] (Application example 1)

[0200] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0201] Traditionally, the process of selecting interior decor and home appliances to create the ideal room when moving into a new home has been extremely time-consuming and laborious. It is particularly difficult to visualize the layout of furniture and home appliances and select the appropriate items, and it is even more time-consuming to search for links to purchase each item one by one. It is also not easy to match the user's desired style with existing furniture. There is a need for a system that can solve these problems and efficiently support the creation of the ideal room.

[0202] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0203] In this invention, the server includes input means for inputting a room image, room dimension information, a list of necessary fixtures and electrical equipment, information on existing fixtures and electrical equipment, and a preferred style from the user, transmission means for transmitting data from the input means to the server, analysis means for receiving the data from the transmission means and performing image analysis and text data analysis, generation means for generating a room layout image using a generation AI based on the results of the analysis means, proposal means for generating a list of suggested fixtures and electrical equipment based on the layout generated by the generation means, link generation means for generating links to purchase pages for the fixtures and electrical equipment proposed by the proposal means, display means for transmitting and displaying the data generated by the link generation means to the user, and link acquisition means for acquiring purchase links for the suggested fixtures and electrical equipment. This allows users to easily generate their ideal room layout and purchase the necessary fixtures and electrical equipment.

[0204] A "user" is a user of the system who inputs information for selecting and arranging interior decor and home appliances for a new home.

[0205] "Room footage" refers to photos and videos of the room that users provide to the system.

[0206] "Room dimension information" is information about the physical size of a room, such as the length, width, and height of the room.

[0207] "Fixtures" refers to the interior and furniture of a room, and usually includes chairs, tables, shelves, etc.

[0208] "Electrical equipment" refers to electrical appliances and devices used in a room, such as a television or refrigerator.

[0209] "Input means" refers to a device or interface that allows a user to input room images, dimensional information, a list of fixtures and electrical equipment, etc. into the system.

[0210] "Transmission means" refers to a device or software for transmitting input data to a server.

[0211] "Analysis means" refers to functions and software for performing image analysis and text data analysis based on received data.

[0212] "Generation means" refers to functions and algorithms for generating a room layout image using generation AI based on the results of the analysis means.

[0213] The "proposal means" is a function or software that creates a list of proposed fixtures and electrical equipment based on the generated layout.

[0214] "Link generation means" refers to the functionality or software for generating links to the purchase pages of the proposed fixtures and electrical equipment.

[0215] "Display means" refers to a device or interface for displaying the generated data, suggested lists, and purchase links to the user.

[0216] "Link acquisition means" refers to the functionality or software for acquiring the purchase links for the proposed fixtures and electrical equipment.

[0217] "Style" refers to the interior design or theme that the user prefers, such as simple modern.

[0218] This invention is a system that assists users in purchasing interior decor and home appliances to create their ideal room when moving into a new home. This system uses generative AI to generate a room layout image based on user input and suggests specific fixtures and electrical equipment. The processing flow of the system's program and its specific operation are explained below.

[0219] System Overview

[0220] This system consists of a terminal and a server. The terminal sends the user's input information to the server and displays the final result to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list.

[0221] Program processing flow and operation

[0222] 1. User Input

[0223] The user uses the terminal to input a picture of the room, the room's dimensions, a list of necessary fixtures and electrical equipment, information on the fixtures and electrical equipment they have, and their preferred style.

[0224] For example, you can input a video of the living room of your new home, the dimensions of the room (5m x 4m), the need for a TV stand and carpet, the sofa and table you already have, and your preferred style (simple modern).

[0225] 2. Data Transmission

[0226] The terminal transmits the collected user input data to the server.

[0227] 3. Data Receipt and Storage

[0228] The server receives the data sent from the device and stores it in a database for analysis.

[0229] 4. Image Analysis

[0230] The server retrieves images of the room from the database and performs image analysis, using image recognition algorithms to identify the layout of the room and the location and dimensions of existing furniture. For example, edge detection and object recognition algorithms are used to identify the contours of the room and the location of furniture.

[0231] 5. Text Data Analysis

[0232] The server analyzes the room dimensions, the list of required fixtures and electrical equipment, and the information on existing fixtures and electrical equipment, thereby understanding the size of the room and the required interior specifications.

[0233] 6. Style Analysis

[0234] The server performs an analysis to identify suitable fixtures and electrical equipment based on the user's preferred style (e.g., simple modern).

[0235] 7. Data integration and input information organization

[0236] The server integrates and organizes the results of image analysis, text data analysis, and style analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[0237] 8. Layout Generation Using Generative AI

[0238] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then uses the user's input information and analysis results to automatically arrange the layout and interior.

[0239] 9. Generate a list of fixtures and electrical equipment

[0240] The server creates a list of suggested furniture and electrical equipment for the user based on the room layout image generated by the AI. For example, it adds a simple, modern TV stand or carpet to the list.

[0241] 10. Generate shopping page links

[0242] The server generates a link to a purchasing page for each proposed fixture or electrical device, possibly using an online shopping API.

[0243] 11. Data Transmission

[0244] The server sends the device an image of the final room layout, a list of proposed fixtures and electrical equipment, and a link to purchase each item.

[0245] 12. Displaying Data

[0246] The terminal displays to the user an image of the final room layout, a list of suggested fixtures and electrical equipment, and a link to purchase.

[0247] 13. User Choices and Purchases

[0248] Users can check the information displayed on their device, select the items they like from the suggested fixtures and electrical equipment, and purchase them using the provided purchase link.

[0249] Hardware and software used

[0250] Hardware:

[0251] Smartphone (with camera function)

[0252] software:

[0253] Python, Pillow, and requests

[0254] Examples:

[0255] User A takes a video of the living room with their smartphone and enters the room dimensions (5m x 4m), a list of necessary furniture (TV stand, carpet), a list of furniture they already have (sofa, table), and their preferred style (simple modern) into an input form.

[0256] The app generates the best simple modern layout image and provides purchase links for the "Simple Modern TV Stand" and "Simple Modern Carpet."

[0257] Example prompts to input to the generative AI model:

[0258] Generate the optimal room layout image based on the user's room image, dimensions, required furniture list, existing furniture information, and preferred style.

[0259] Room video: <Video from user>

[0260] Room dimensions: 5m long, 4m wide

[0261] List of necessary furniture: TV stand, carpet

[0262] Current fixtures: sofa, table

[0263] Favorite style: Simple modern

[0264] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0265] Step 1:

[0266] The user uses a device to input a video of the room, information about the room's dimensions, a list of the necessary fixtures and electrical equipment, information about the fixtures and electrical equipment they currently have, and their preferred style. A smartphone application is used as the input method. The data the user inputs might include, for example, a video of the living room in their new home, the dimensions of the room (5m long x 4m wide), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style being simple and modern. This input information is saved as input data.

[0267] Step 2:

[0268] The terminal sends the collected user input data to the server. This transmission method uses the HTTP protocol via the Internet. The input data is converted into JSON format and sent to the server. This data transmission procedure passes data related to the user's needs to the server.

[0269] Step 3:

[0270] The server receives the data sent from the device and stores it in a database for analysis. The server, as the receiving means, verifies the accuracy of the data and stores it in the database if there are no problems. The stored data is necessary for subsequent analysis processing and serves as the basis for realizing the user's wishes.

[0271] Step 4:

[0272] The server retrieves images of the room from the database and performs image analysis. Image analysis determines the room layout and the location and dimensions of existing furniture. Specifically, edge detection and object recognition algorithms are used to identify the contours of the room and the location of furniture. The image of the room is used as input data, and data on the layout and dimensions of furniture is output.

[0273] Step 5:

[0274] The server analyzes the room's dimensions, a list of required fixtures and electrical equipment, and information on existing fixtures and electrical equipment. It then analyzes the text data to understand the room's size and required interior specifications. This information is used as input data, and the specific specifications needed for fixture placement and selection are output.

[0275] Step 6:

[0276] The server performs an analysis to identify suitable fixtures and electrical equipment based on the user's preferred style. Analysis of the style data allows for the selection of fixtures based on the user's preferred taste, such as simple modern. Style information is used as input data, and fixtures and electrical equipment that match the user's preferences are output.

[0277] Step 7:

[0278] The server integrates and organizes the results of image analysis, text data analysis, and style analysis. This clarifies the design conditions that are optimal for the user's wishes and the characteristics of the room. Multiple analysis results are integrated and the optimal layout conditions are output.

[0279] Step 8:

[0280] The server calls the generation AI based on the integrated data and generates an optimal room layout image. The generation AI uses the user's input information and analysis results to automatically arrange the layout and interior. The following text is used as an input prompt:

[0281] Generate the optimal room layout image based on the user's room image, dimensions, required furniture list, existing furniture information, and preferred style.

[0282] Room video: <Video from user>

[0283] Room dimensions: 5m long, 4m wide

[0284] List of necessary furniture: TV stand, carpet

[0285] Current fixtures: sofa, table

[0286] Favorite style: Simple modern

[0287] Step 9:

[0288] The server creates a list of furniture and electrical equipment to suggest to the user based on the room layout image generated by the AI. A list of appropriate furniture and electrical equipment is output from the generated layout image, allowing the user to select the optimal interior.

[0289] Step 10:

[0290] The server generates a link to the purchase page for each proposed fixture or electrical device. The link generation method uses an online shopping API, which outputs a link that allows the user to directly access the purchase page.

[0291] Step 11:

[0292] The server sends the final room layout image, a list of proposed fixtures and electrical equipment, and a link to purchase each product to the terminal. The data is sent via the HTTP protocol over the Internet. The data output contains all the necessary information for the user.

[0293] Step 12:

[0294] The terminal displays the final room layout image, a list of suggested fixtures and electrical equipment, and a link to purchase. The user can review this information and evaluate the room layout. Visual information is output for optimal interior selection.

[0295] Step 13:

[0296] Users can check the information displayed on their devices, select the items they like from the suggested fixtures and electrical equipment, and purchase them using the provided purchase link. This simplifies the selection and purchase process, providing users with concrete steps to create their ideal room.

[0297] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0298] The present invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room for their new home. By incorporating an emotion engine that recognizes the user's emotions, the system can provide more appropriate suggestions based on the user's emotional state. The processing flow and specific operations of the system's program are explained below.

[0299] System Overview

[0300] This system consists of a terminal and a server. The terminal sends the user's input information to the server, analyzes the user's emotions through an emotion engine, and displays the final results to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list, adjusting them based on the user's emotional state.

[0301] Program processing flow and operation

[0302] 1. User Input

[0303] The user uses a terminal to input a photo of the room, information about the room's size, a list of the furniture and appliances they need, information about the furniture and appliances they already have, and their preferred style.The emotion engine then analyzes the user's voice, facial expression, and input speed in real time to grasp the user's emotional state.For example, a user might input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the fact that they already have a sofa and table, and their preferred style (simple modern).

[0304] 2. Data Transmission

[0305] The terminal transmits the data input by the user and the emotion data analyzed by the emotion engine to the server.

[0306] 3. Data Receipt and Storage

[0307] The server receives the data sent from the device and stores it in a database for analysis.

[0308] 4. Image Analysis

[0309] The server retrieves the room's photo data from the database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. For example, edge detection algorithms and object recognition techniques are used to detect the contours of the room and the location of furniture.

[0310] 5. Text Data Analysis

[0311] The server analyzes the text data provided by the user, including the room size, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, to understand the room size and required interior specifications.

[0312] 6. Taste and Emotion Analysis

[0313] The server performs analysis to identify suitable furniture and appliances based on the user's preferred taste (e.g., simple modern). It also adjusts the taste and style of the furniture and appliances it recommends based on the user's emotional state received from the emotion engine. For example, if the user is feeling stressed, it will suggest interior items that will help them relax.

[0314] 7. Data integration and input information organization

[0315] The server integrates and organizes the results of image analysis, text data analysis, taste analysis, and emotion analysis from the emotion engine, thereby clarifying the optimal design conditions for the user's wishes and the characteristics of the room.

[0316] 8. Layout Generation Using Generative AI

[0317] The server then calls a generative AI based on the integrated data to generate an optimal room layout image. This generative AI automatically arranges furniture and appliances to create the optimal layout based on the user's input data, analysis results, and emotional state.

[0318] 9. Generate a list of interior, furniture, and home appliances

[0319] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a TV stand or carpet with a simple, modern design to the list. The server also takes into account the user's emotional state to further optimize the options.

[0320] 10. Generate shopping page links

[0321] The server generates a link to the purchase page for each suggested piece of furniture or home appliance, taking into account the size and placement information of each product. Based on the analysis results of the emotion engine, the server makes suggestions that will increase user satisfaction.

[0322] 11. Data Transmission

[0323] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[0324] 12. Displaying Data

[0325] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[0326] 13. User Choices and Purchases

[0327] Users can select the furniture or appliances they like from the suggested list displayed on their device and complete the purchase using the provided link. At this time, they can also check the specific interior layout while making their selection.

[0328] This concludes the description of the embodiment of the invention. This system not only allows users to efficiently and effectively select and arrange furniture and home appliances that suit their room and preferences, but also provides links to purchase each item. Furthermore, by adjusting the suggestions based on the user's emotional state, it is possible to create a room that is more satisfying.

[0329] The processing flow will be explained below.

[0330] Step 1:

[0331] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they currently have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style (simple modern).

[0332] Step 2:

[0333] The device recognizes the user's emotional state through the user's tone of voice, facial expression, and input speed. The emotion engine analyzes this information and estimates the user's emotional state, for example, determining whether the user is relaxed or nervous.

[0334] Step 3:

[0335] The terminal transmits all data input by the user and emotion data analyzed by the emotion engine to the server.

[0336] Step 4:

[0337] The server receives the data sent from the device and stores it in a database for analysis.

[0338] Step 5:

[0339] The server retrieves the room's photo data from the database and performs image analysis, using edge detection algorithms and object recognition techniques to identify the room layout and the location and dimensions of existing furniture. For example, it recognizes the boundaries between the floor and walls of the room and identifies the placement of existing furniture.

[0340] Step 6:

[0341] The server analyzes the room size information, the list of necessary furniture and appliances, and the information on the furniture and appliances currently available as text data. This clarifies the specific dimensions of the room and the interior requirements. For example, it determines the size of the TV stand and carpet that are appropriate for the living room.

[0342] Step 7:

[0343] The server analyzes the user's preferred taste (e.g., simple modern) and identifies suitable furniture and home appliances. It also adjusts the taste and style of the furniture and home appliances it recommends based on the emotional data obtained from the emotion engine. For example, if the user wants to relax, it will suggest furniture in soft colors.

[0344] Step 8:

[0345] The server integrates the results of image analysis, text data analysis, taste analysis, and emotion analysis from the emotion engine to determine optimal design conditions.

[0346] Step 9:

[0347] The server uses the integrated data to call a generative AI to generate an optimal room layout image. This generative AI automatically arranges furniture and appliances based on the user's input data and analysis results, and creates the optimal layout by taking emotional data into consideration.

[0348] Step 10:

[0349] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a TV stand or carpet with a simple, modern design to the list, reflecting the results of sentiment analysis.

[0350] Step 11:

[0351] The server generates links to purchase pages for the suggested furniture and appliances, taking into account the size and placement of each product and adjusting them according to the user's emotional state.

[0352] Step 12:

[0353] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[0354] Step 13:

[0355] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[0356] Step 14:

[0357] Users can select their favorite furniture or appliances from the suggested list displayed on their device and purchase them using the provided link, or they can rearrange the proposed interior. Here, they can make their final selection while checking the specific interior layout.

[0358] Example 2

[0359] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0360] Conventional purchasing support systems for room interiors and home appliances require users to manually sort through large amounts of information and determine the optimal interior layout, which is extremely time-consuming. Furthermore, since suggestions are not based on the user's emotional state, this can result in low user satisfaction. There is a need for a system that overcomes these drawbacks, reduces the user's time and effort, and provides suggestions that are in line with individual emotions.

[0361] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0362] In this invention, the server includes: input means for inputting image data of the room, room dimension information, a list of necessary items, information on items on hand, and a preferred interior style from the user; communication means for transmitting the data from the input means to a central processing unit; analysis means for receiving the data from the communication means and performing image analysis and text data analysis; emotion recognition means for grasping the user's emotional state; generation means for generating a room layout image using a generative AI model based on the results of the analysis means and the emotion recognition means; suggestion means for generating a list of suggested items based on the layout generated by the generation means; link generation means for generating links to purchase pages for the items suggested by the suggestion means; and display means for transmitting the data generated by the link generation means to the user and displaying it. This allows the user to receive suggestions for optimal interior layout and items based on their emotional state.

[0363] "User" refers to a person who uses this system to receive suggestions for room interiors and home appliances.

[0364] "Input means" refers to a device or software that allows a user to input image data of a room, dimensional information, a list of necessary items, information on items on hand, and a preferred interior style.

[0365] "Communication means" refers to a method or device for transmitting data from an input means to a central processing unit.

[0366] "Analysis means" refers to devices and algorithms that perform image analysis and text data analysis based on received data.

[0367] "Emotion recognition means" refers to devices or software that analyze a user's voice, facial expressions, input speed, etc. to understand their emotional state.

[0368] "Generation means" refers to a device or software for generating a room layout image using a generative AI model based on the results of the analysis means and emotion recognition means.

[0369] The "suggestion means" refers to a method or apparatus for creating a suggested item list based on the layout generated by the generation means.

[0370] The "link generating means" refers to a method or device for generating a link to a purchase page for the item suggested by the suggesting means.

[0371] The "display means" refers to a device or software for transmitting the data generated by the link generation means to the user and visually presenting it to the user.

[0372] A "generative AI model" refers to an artificial intelligence algorithm that automatically creates the optimal room layout based on user input data and analysis results.

[0373] The present invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room when moving into a new home. This system is configured using a terminal and a server. The detailed configuration and operation of the system are described below.

[0374] overview

[0375] This system assists users in the process of selecting the best interior and home appliances for their new home. The system incorporates an emotion engine to constantly grasp the user's emotional state and make more appropriate suggestions. Users can input data using their device to review and purchase the suggested interior and home appliances.

[0376] Specific actions

[0377] 1. User Input

[0378] The user uses the device to input image data of the room they will be living in, the room's dimensions, a list of items they need, information about items they already have, and their preferred interior style. The emotion engine analyzes the user's voice, facial expressions, and input speed in real time to grasp their emotional state.

[0379] Example: Take a photo of the living room of your new home with your smartphone and upload it. Then, enter the room dimensions (5m x 4m), items you need (TV stand, carpet), furniture you already have (sofa, table), and your preferred interior style (simple modern) in the input fields.

[0380] 2. Data Transmission

[0381] The terminal transmits data input by the user and emotion data analyzed by the emotion engine to the server. This data includes images, text, and emotion analysis results.

[0382] 3. Data Receipt and Storage

[0383] The server receives the data sent from the device and stores it in a database for analysis. The stored data includes photos of the room, text data, and emotion analysis data.

[0384] 4. Image Analysis

[0385] The server retrieves photos of the room from a database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. Techniques used include edge detection and object recognition algorithms.

[0386] Example: The server uses the image analysis library OpenCV to extract the contours of a room and the position of furniture using an edge detection algorithm.

[0387] 5. Text Data Analysis

[0388] The server performs text analysis of the room dimension information, list of required items, and information on items on hand provided by the user to understand the size of the room and the specifications of the required items.

[0389] 6. Taste and Emotion Analysis

[0390] The server performs analysis to identify suitable items based on the user's preferred interior style (e.g., simple modern), and adjusts the style of the suggested items based on the user's emotional state received from the emotion engine.

[0391] Example: If the user is feeling stressed, suggest interior design that will help them relax.

[0392] 7. Data integration and input information organization

[0393] The server integrates and organizes the image analysis results, text data analysis results, taste analysis results, and emotion analysis results from the emotion engine.

[0394] 8. Layout Generation Using Generative AI Models

[0395] The server calls up a generative AI model based on the integrated data and generates an optimal room layout image.

[0396] Example prompt for a generative AI model:

[0397] "Generate a room layout in a simple, modern interior style using the user's input data: room dimensions (5m x 4m), items on hand (sofa and table), and a list of items needed (TV stand and carpet). The user's emotional state should be relaxed."

[0398] 9. Generate a list of interior items

[0399] The server creates a list of items to suggest to the user based on the room layout image generated by the generative AI model.

[0400] 10. Generate shopping page links

[0401] The server generates a link to the purchase page for each suggested item, taking into account the analysis results of the emotion engine to ensure a high level of user satisfaction.

[0402] 11. Data Transmission

[0403] The server sends the generated final data (room layout image, suggested item list, and purchase link) to the terminal.

[0404] 12. Displaying Data

[0405] The terminal displays the final room layout image, a list of suggested items, and a link to purchase them to the user, allowing the user to select the interior design while reviewing them.

[0406] 13. User Choices and Purchases

[0407] Users can select the items they like from the suggested list displayed on their device and complete the purchase using the provided link, while also checking the specific interior layout.

[0408] This system not only allows users to efficiently and effectively select and arrange interior and home appliances that suit their room and preferences, but also provides links to purchase each item. Furthermore, by adjusting the suggestions based on the user's emotional state, it is possible to create a room that is more satisfying.

[0409] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0410] Step 1:

[0411] User Input

[0412] The user uses the device to input image data of the room, room dimensions, a list of items needed, information about items they have on hand, and their preferred interior style. The emotion engine analyzes voice, facial expressions, and input speed in real time to grasp the user's emotional state.

[0413] Input: room image data, room dimensions, list of necessary items, information on items on hand, preferred interior style

[0414] Output: User input data and emotion data stored on the device

[0415] How it works: The user takes a photo of the living room of their new home with their smartphone and uploads it to the application. Then, they fill in the input fields with the room dimensions (5m x 4m), items they need (TV stand, carpet), furniture they already have (sofa, table), and their preferred interior style (simple modern).

[0416] Step 2:

[0417] Data transmission

[0418] The terminal transmits data input by the user and emotion data analyzed by the emotion engine to the server.

[0419] Input: User input data and emotion data stored on the device

[0420] Output: User input data and emotion data sent to the server

[0421] Specific operation: The terminal uses the API to send input data in JSON format to the server.

[0422] Step 3:

[0423] Data reception and storage

[0424] The server receives the data sent from the device and stores it in a database for analysis.

[0425] Input: User input data and emotion data sent to the server

[0426] Output: User data and emotion data stored in a database

[0427] Specific operation: The server calls the database save function to save the received data. For example, the image data of the room is saved in Blob format, and the text data is saved in a structured database.

[0428] Step 4:

[0429] Image analysis

[0430] The server retrieves image data of the room from the database and performs image analysis to determine the room layout, the location of existing furniture, and its dimensions.

[0431] Input: Image data of the room stored in the database

[0432] Output: Room layout, existing furniture locations, dimensions

[0433] Specific operation: The server uses the OpenCV library to extract the room contours and furniture positions using an edge detection algorithm.

[0434] Step 5:

[0435] Text Data Analysis

[0436] The server performs text analysis of the room dimension information, list of required items, and information on items on hand provided by the user to understand the size of the room and the specifications of the required items.

[0437] Input: Room dimensions stored in the database, list of required items, and information on items on hand

[0438] Output: Analyzed room size, required item specifications

[0439] What it does: The server uses natural language processing (NLP) tools to parse the text input data and get the room dimensions and a list of required items.

[0440] Step 6:

[0441] Taste and emotion analysis

[0442] The server performs analysis to identify suitable items based on the user's preferred interior style (e.g., simple modern), and adjusts the style of the suggested items based on the user's emotional state received from the emotion engine.

[0443] Input: User's preferred interior style, sentiment analysis results

[0444] Output: Adjusted product list and style suggestions

[0445] How it works: The server uses a taste analysis algorithm to search for furniture that suits a simple modern style, and customizes suggestions based on the results of sentiment analysis.

[0446] Step 7:

[0447] Data integration and input organization

[0448] The server integrates and organizes the results of image analysis, text data analysis, taste analysis, and emotion analysis.

[0449] Input: Image analysis results, text data analysis results, taste analysis results, emotion analysis results

[0450] Output: Integrated interior proposal data

[0451] Specific operation: The server integrates these analysis results into a single data structure, compiling all the information necessary to propose the optimal interior design.

[0452] Step 8:

[0453] Layout generation using generative AI models

[0454] The server calls up a generative AI model based on the integrated data and generates an optimal room layout image.

[0455] Input: Integrated interior proposal data

[0456] Output: Generated room layout image

[0457] Specific operation: The server sends a prompt to the generative AI model (e.g., "Generate a simple, modern living room layout based on the user's input data") and retrieves the returned results.

[0458] Step 9:

[0459] Generate a list of interior items

[0460] The server creates a list of items to suggest to the user based on the room layout image generated by the generative AI model.

[0461] Input: Generated room layout image

[0462] Output: List of suggested items

[0463] How it works: The server checks a database of furniture and appliances to create a list of items that match the layout image.

[0464] Step 10:

[0465] Generate shopping page links

[0466] The server generates a link to a purchase page for each suggested item, taking into account the analysis results of the emotion engine to maximize user satisfaction.

[0467] Input: List of suggested items, sentiment analysis results

[0468] Output: Link to purchase page

[0469] What it does: The server collects online shop links for each item and adds the links to a list for easy user access.

[0470] Step 11:

[0471] Sending data

[0472] The server sends the generated final data (room layout image, suggested item list, and purchase link) to the terminal.

[0473] Input: Final data (room layout image, suggested item list, purchase link)

[0474] Output: Final data sent to the terminal

[0475] Specific operation: The server uses the API to send the final data to the terminal.

[0476] Step 12:

[0477] Viewing Data

[0478] The terminal displays the final room layout image, a list of suggested items, and a link to purchase them to the user, allowing the user to select the interior design while reviewing them.

[0479] Input: Final data sent from the server

[0480] Output: Layout image displayed to the user, item list, purchase link

[0481] Specific operation: The device displays the acquired data on the application's UI, making it easy for the user to view.

[0482] Step 13:

[0483] User Choice and Purchase

[0484] The user selects the item they like from a list of suggestions displayed on their device and completes the purchase using the provided link.

[0485] Input: Displayed suggestion list and purchase link

[0486] Output: Purchased item

[0487] Specific operation: The user clicks on the purchase link, goes to the online shop page and purchases the product.

[0488] This allows users to efficiently select and arrange interior and home appliances that suit their room and preferences, and receive suggestions based on their emotional state.

[0489] (Application example 2)

[0490] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0491] Conventional interior design suggestion systems have difficulty in making suggestions that take into account the user's emotional state, which results in a failure to increase user satisfaction. Furthermore, when purchasing in a physical store, they are unable to make optimal suggestions based on real-time interior layout or emotions, which means it takes a lot of time and effort for users to realize their ideal room.

[0492] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0493] In this invention, the server includes: emotion analysis means for analyzing the user's emotional state and adjusting the proposal content based on that state; input means for inputting from the user an image of the room, room dimension information, a list of necessary household goods and home appliances, information on household goods and home appliances currently in use, and a preferred design; transmission means for transmitting data from the input means to a central processing unit; analysis means for receiving the data from the transmission means and performing image analysis and text data analysis; generation means for generating a room layout image using a generation AI based on the results of the analysis means; proposal means for generating a list of proposed household goods and home appliances based on the layout generated by the generation means; link generation means for generating links to purchasing pages for the household goods and home appliances proposed by the proposal means; and display means for transmitting and displaying the data generated by the link generation means to the user. This enables optimal interior proposals in real time while taking the user's emotional state into consideration, and facilitates smooth purchases in physical stores.

[0494] "User" refers to a person who uses the system.

[0495] "Room image" refers to a photo of a room taken by a user.

[0496] "Room dimension information" refers to information about the length, width, and height of a room.

[0497] "List of necessary household goods and home appliances" refers to a list of household goods and home appliances that the user wants to purchase.

[0498] "Information about existing household goods and home appliances" refers to information about household goods and home appliances that the user already owns.

[0499] "Preferred design" refers to the interior style or theme that the user prefers.

[0500] "Input means" refers to a device or method by which a user inputs information into a system.

[0501] The "transmission means" refers to a function for transmitting data acquired from the input means to the central processing unit.

[0502] "Analysis means" refers to a device or method that analyzes received data and performs image and text data analysis.

[0503] "Generation means" refers to the function of generating a room layout image using generation AI based on the results of the analysis means.

[0504] The "suggestion means" refers to a function for generating a list of suggested household goods and home appliances based on the arrangement generated by the generation means.

[0505] The "link generating means" refers to a function for generating a link to a purchase page for the household goods / home appliances suggested by the suggesting means.

[0506] The "display means" refers to a function that displays the data generated by the link generation means to the user.

[0507] "Emotion analysis means" refers to a function that analyzes the user's emotional state and adjusts the content of suggestions based on that state.

[0508] "Central Processing Unit" refers to the main unit for processing, analyzing and generating user input data.

[0509] The present invention is a system that supports the purchase of interior decorations and home appliances to realize the ideal room when a user moves into a new home. By combining this system with emotion analysis means, it is possible to make more appropriate suggestions according to the user's emotional state.

[0510] System configuration

[0511] This system is mainly composed of terminals and servers, and is explained in detail below.

[0512] Terminal

[0513] The terminal is a device that inputs information through user operation and sends it to the server. The terminal is equipped with hardware such as a camera, microphone, and display, and performs the following functions:

[0514] Input means: The user uses the terminal to input an image of the room, room dimensions, a list of necessary household goods and appliances, information about the household goods and appliances they already have, and their preferred design.

[0515] Emotion analysis means: When a user inputs information, voice, facial expressions, input speed, etc. are analyzed in real time to understand the user's emotional state.

[0516] Transmission method: The input data and the emotion analysis results are sent to the server.

[0517] server

[0518] The server receives data sent from the terminal, analyzes it, generates it, and makes suggestions. It has the following functions:

[0519] Analysis methods: Image analysis and text data analysis are performed. Image analysis uses edge detection algorithms and object recognition techniques to identify the room layout and the location of existing furniture. Text data analysis analyzes the room dimensions and specifications of required furniture.

[0520] Generation method: Based on the analysis results, a generative AI is used to generate an optimal room layout image.

[0521] Proposal method: Based on the generated layout image, a list of suggested household goods and home appliances is generated.

[0522] Link generation method: Generate a link to the purchase page for the suggested household goods and appliances.

[0523] Display means: The generated data is sent to the terminal and displayed to the user.

[0524] Detailed System Operation

[0525] Program processing

[0526] The server uses image processing libraries such as OpenCV to analyze the room photos sent by the user. It also uses a natural language processing engine to analyze text data. It uses a generative AI model (such as GAN or VAE) to generate an optimal room layout image based on the analysis results.

[0527] Emotion analysis means

[0528] The emotion analysis method uses an emotion recognition engine (such as Microsoft Azure's Emotion API or Google Cloud's Vision API) to analyze the user's emotional state in real time from their voice and facial expressions. This allows the system to suggest interior designs that will help the user relax if they are feeling stressed, or that will help them maintain their excitement if they are enjoying themselves.

[0529] Specific examples

[0530] For example, suppose a user takes a photo of the living room of their new home and inputs the room dimensions (5m x 4m), the furniture they need (TV stand and carpet), the furniture they already have (sofa and table), and their preferred design (simple modern). Based on this information and the results of sentiment analysis, the server uses a generative AI model to generate an optimal layout image. It then creates a list of suggested furniture and appliances and generates a link to the purchasing page to display it to the user.

[0531] Prompt Sentence Examples

[0532] "Generate an optimal room layout image based on a photo of the room, size information, a list of necessary furniture and appliances, information on the furniture and appliances currently in use, and the user's preferred design (e.g., simple modern). Please also include suggestions based on the user's emotional state (e.g., relaxation)."

[0533] This system not only allows users to efficiently and effectively select and arrange furniture and appliances that suit their room and preferences, but also provides links to purchase each item. Furthermore, by adjusting the suggestions based on the user's emotional state, it is possible to create a room that is more satisfying.

[0534] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0535] Step 1:

[0536] The user uses the terminal to input a picture of the room, dimension information, a list of necessary household goods and appliances, information about the household goods and appliances they already have, and their preferred design.

[0537] Input: Room photo, dimensions, list of necessary furniture and appliances, information on existing furniture and appliances, preferred design

[0538] Output: Input data (a series of information)

[0539] Specific operation: The user uses the application on their smartphone or tablet to take a photo of the room using the camera function and fill out an input form with various information. This information is temporarily stored on the device as integrated data.

[0540] Step 2:

[0541] The terminal uses an emotion analysis means to analyze the user's emotional state in real time from their voice and facial expressions.

[0542] Input: User's voice, facial expressions

[0543] Output: Sentiment analysis data

[0544] How it works: The device uses a microphone and camera to capture the user's voice and facial expressions. This data is then fed into an emotion analysis engine (e.g., Emotion API) to analyze the user's emotional state (e.g., stress, joy) in real time.

[0545] Step 3:

[0546] The terminal transmits the input data and the emotion analysis results to the server.

[0547] Input: Input data, sentiment analysis data

[0548] Output: Data sent to the server

[0549] Specific operation: The device combines the user's input data and emotion analysis data and sends it to the server as a single data packet. The communication protocol is HTTPS or similar, ensuring secure transmission.

[0550] Step 4:

[0551] The server stores the received data in a database for analysis and performs image analysis and text data analysis.

[0552] Input: Data sent to the server

[0553] Output: Image analysis results, text data analysis results

[0554] Specific operation: The server stores the received data in an analysis database. It then analyzes the room photo using image processing libraries such as OpenCV, and performs edge detection algorithms and object recognition. At the same time, it uses a text data analysis engine to analyze the dimensions and furniture list.

[0555] Step 5:

[0556] Based on the analysis results, the server uses a generative AI model to generate an optimal room layout image.

[0557] Input: Image analysis results, text data analysis results, emotion analysis data

[0558] Output: Generated aligned image

[0559] How it works: The server takes the analysis results and emotion data as input and provides them to a generative AI model (e.g., GAN), which then generates an optimal room layout image that matches the user's preferences.

[0560] Step 6:

[0561] The server generates a list of suggested household goods and home appliances based on the generated layout image.

[0562] Input: Generated aligned image

[0563] Output: List of suggested household goods and appliances

[0564] How it works: The server analyzes the layout images output by the generative AI model and automatically generates a list of optimal proposals, including detailed information such as the size and design of furniture and appliances.

[0565] Step 7:

[0566] The server generates a link to the purchase page for the suggested household goods and home appliances.

[0567] Input: List of suggested household goods and appliances

[0568] Output: Link to purchase page

[0569] How it works: For each item in the suggestion list, the server searches an online shopping database and generates a link to the page where it can be purchased, optimized to take into account the user's emotional state.

[0570] Step 8:

[0571] The server transmits the generated data to the terminal, which displays it to the user.

[0572] Input: Purchase link, proposal list, placement image

[0573] Output: What is displayed to the user

[0574] Specific operation: The server sends a link to the purchase page, a list of suggestions, and layout images to the device. The device receives this data and displays it in a format that is easy for the user to view. The user can then select and purchase interior items based on the displayed information.

[0575] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0576] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0577] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0578] [Second embodiment]

[0579] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0580] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0581] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0582] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0583] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0584] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0585] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0586] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0587] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0588] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0589] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0590] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0591] This invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room when moving into a new home. This system uses a generative AI to generate a room layout image based on user input and suggests specific interior decor and home appliances. The processing flow of the system's program and its specific operation are explained below.

[0592] System Overview

[0593] This system consists of a terminal and a server. The terminal sends the user's input information to the server and displays the final result to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list.

[0594] Program processing flow and operation

[0595] 1. User Input

[0596] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they currently have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style (simple modern).

[0597] 2. Data Transmission

[0598] The terminal transmits the collected user input data to the server.

[0599] 3. Data Receipt and Storage

[0600] The server receives the data sent from the device and stores it in a database for analysis.

[0601] 4. Image Analysis

[0602] The server retrieves a photo of the room from the database and performs image analysis, using image recognition algorithms to identify the room's layout and the location and dimensions of existing furniture, for example, using edge detection and object recognition algorithms to identify the contours of the room and the location of furniture.

[0603] 5. Text Data Analysis

[0604] The server analyzes the room size information, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, thereby understanding the size of the room and the required interior specifications.

[0605] 6. Taste Analysis

[0606] The server performs an analysis to identify suitable furniture and appliances based on the user's preferred style (e.g., simple modern).

[0607] 7. Data integration and input information organization

[0608] The server integrates and organizes the results of image analysis, text data analysis, and taste analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[0609] 8. Layout Generation Using Generative AI

[0610] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then uses the user's input information and analysis results to automatically arrange the layout and interior.

[0611] 9. Generate a list of interior, furniture, and home appliances

[0612] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a simple, modern TV stand or carpet to the list.

[0613] 10. Generate shopping page links

[0614] The server generates a link to a purchasing page for each suggested piece of furniture or appliance, possibly using an online shopping API.

[0615] 11. Data Transmission

[0616] The server sends the device an image of the final room layout, a list of suggested furniture and appliances, and a link to purchase each item.

[0617] 12. Displaying Data

[0618] The device displays to the user an image of the final room layout, a list of suggested furniture and appliances, and a link to purchase.

[0619] 13. User Choices and Purchases

[0620] Users can check the information displayed on their device, select the furniture or home appliances they like from the suggested items, and purchase them using the provided purchase link.

[0621] This concludes the description of the embodiment of the invention. This system not only allows users to efficiently and effectively select and arrange furniture and home appliances that suit their room and preferences, but also provides links to purchase each item at the same time. This allows users to easily create their ideal room.

[0622] The processing flow will be explained below.

[0623] Step 1:

[0624] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they already have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), that they need a TV stand and carpet, that they already have a sofa and table, and that their preferred style is simple and modern.

[0625] Step 2:

[0626] The terminal collectively transmits the data input by the user to the server.

[0627] Step 3:

[0628] The server receives the data sent from the device and stores it in a database for analysis.

[0629] Step 4:

[0630] The server retrieves the room's photo data from the database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. For example, edge detection algorithms and object recognition techniques are used to detect the contours of the room and the location of furniture.

[0631] Step 5:

[0632] The server analyzes the text data provided by the user, including the room size, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, to understand the room size and required interior specifications.

[0633] Step 6:

[0634] The server performs analysis to identify suitable furniture and appliances based on the user's preferred style (e.g., simple modern).

[0635] Step 7:

[0636] The server integrates and organizes the results of image analysis, text data analysis, and taste analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[0637] Step 8:

[0638] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then automatically arranges furniture and appliances based on the user's input data and analysis results to create the optimal layout.

[0639] Step 9:

[0640] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the AI. For example, it adds a TV stand or carpet with a simple, modern design to the list.

[0641] Step 10:

[0642] The server generates a link to the online shopping site for each proposed piece of furniture or appliance, taking into account the size and placement information of each item.

[0643] Step 11:

[0644] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[0645] Step 12:

[0646] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[0647] Step 13:

[0648] Users can select the furniture or appliances they like from the suggested list displayed on their device and complete the purchase using the provided link. At this time, they can also check the specific interior layout while making their selection.

[0649] Example 1

[0650] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0651] In conventional interior and home appliance purchasing support systems, it has been difficult for users to effectively plan the layout, interior, and placement of home appliances in a room. In particular, there was a lack of a way to generate an optimal layout while taking into account the user's preferences, existing furniture, and room size. In addition, generating purchase links had to be done manually, which was a cumbersome and time-consuming task for users. These challenges made it difficult to efficiently create the ideal room.

[0652] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0653] In this invention, the server includes input means for allowing a user to input a photo of the room, room dimension information, a list of necessary fixtures, information about fixtures on hand, and a preferred design style, transmission means for transmitting data from the input means to the server, analysis means for receiving the data from the transmission means and performing image analysis and text data analysis, generation means for generating a room layout image using a generative AI model based on the results of the analysis means, proposal means for generating a proposed fixture list based on the layout generated by the generation means, link generation means for generating links to purchasing pages for the fixtures proposed by the proposal means, and display means for transmitting and displaying the data generated by the link generation means to the user. This enables users to efficiently design a room with an ideal layout based on their preferences and easily purchase the necessary fixtures.

[0654] A "user" is an individual or organization that uses the system to receive support for room layout, interior design, and purchasing home appliances.

[0655] A "terminal" is a device used by a user to provide input information to the system, and includes a PC, smartphone, tablet, etc.

[0656] The "server" is a central processing unit that receives data sent from users, analyzes it, and generates layout images and proposal lists.

[0657] The "input means" is an interface that allows the user to input a photo of the room, dimension information, a list of necessary equipment, information on equipment that is on hand, and a preferred design style.

[0658] The "transmission means" is a means for transmitting the user's input data from the terminal to the server.

[0659] The "analysis means" is a means for analyzing received data in the server and performing image analysis and text data analysis.

[0660] The "generation means" is a means for generating a room layout image using a generative AI model based on the results of the analysis means.

[0661] The "proposing means" is a means for generating an equipment list to be proposed to the user based on the layout generated by the generating means.

[0662] The "link generating means" is a means for generating a link to a purchasing page for the equipment suggested by the suggesting means.

[0663] The "display means" is a means for transmitting the data generated by the link generation means to the user and displaying it.

[0664] "Image analysis" is a technology that processes photos of a room sent by a user and identifies the outline of the room and the location of furniture.

[0665] "Text data analysis" is a technology that analyzes the room dimension information, required equipment list, equipment information on hand, and preferred design style sent by the user.

[0666] A "generative AI model" is an artificial intelligence model that generates optimal room layout images based on user input information and analysis results.

[0667] A "prompt sentence" is an input sentence used to give instructions to a generative AI model.

[0668] "Fixtures" is a general term for decorative and functional items, including interior furnishings and home appliances.

[0669] "Design style" is a concept that refers to a particular theme or aesthetic of room decoration or interior that a user prefers.

[0670] This invention is a system that efficiently supports the purchase of interior decor and home appliances to create an ideal room when a user moves into a new residence. This system is composed of a terminal and a server. The terminal sends the user's input information to the server, and the server uses a generative AI model to generate a room layout image and a proposal list. The final data is then provided to the user.

[0671] Input Method

[0672] Users use their device to input a photo of their room, its dimensions, a list of necessary furnishings, information about the furnishings they already have, and their preferred design style. For example, a user can take a photo of their living room with their smartphone and upload it to the application. They can also input the room dimensions (5 meters long and 4 meters wide), select a TV stand and carpet as necessary items, input the sofa and table they already have, and set their preferred design style to simple modern.

[0673] Transmission method

[0674] The terminal sends the user's input data collected through the input means to the server. This transmission is securely performed using the HTTPS protocol. The input data is converted to JSON format, encrypted, and sent to the server.

[0675] Analysis means

[0676] The server receives data sent from the device and stores it in a database for analysis. For image analysis, the open-source image processing library "OpenCV" is used, using edge detection and object recognition algorithms to identify the outline of the room and the location of furniture. For text data analysis, the "NLTK" library is used to analyze room dimensions and a list of necessary equipment. For taste analysis, generative AI models such as "GPT-3" are used.

[0677] generation means

[0678] Based on the analysis results, the server uses a generative AI model to generate a room layout image. Specifically, it uses a generative AI model such as "DALL·E" and generates the optimal room layout image by inputting a text prompt. The following text is an example of a prompt that can be used:

[0679] Example prompt sentence:

[0680] User input details:

[0681] Room photo

[0682] Room size: 5 meters by 4 meters

[0683] Required furniture and appliances: TV stand and carpet

[0684] Existing items: Sofa and table

[0685] Preferred style: Simple Modern

[0686] Please generate an optimal room layout image and suggest a list of furniture and appliances.

[0687] Proposal means

[0688] The server then creates a list of suggested furnishings for the user based on the generated layout image. For example, it adds a TV stand and carpet suitable for a simple modern style to the list. This suggestion is based on the analyzed user preferences.

[0689] Link Generation Methods

[0690] For each proposed item, the server generates a link to an online shopping site to purchase the item. Specifically, it uses the API of a popular online marketplace to retrieve the product link.

[0691] Display means

[0692] The server sends the final room layout image, the proposed equipment list, and a purchase link for each item to the terminal, which then displays this information to the user. For example, the user can view the layout image and the proposed equipment list through the application interface and click the purchase link to go to a shopping site.

[0693] User Choice and Purchase

[0694] Users can select their favorite items based on the suggested information and complete the purchase process using the provided purchase link, allowing them to efficiently create their ideal room.

[0695] The system of the present invention allows users to simultaneously obtain layout suggestions that suit their room and preferences, a list of necessary fixtures, and links to purchase each item, making it easy for users to create their ideal room.

[0696] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0697] Step 1:

[0698] The user uses the terminal to input a photo of the room, room dimensions, a list of necessary equipment, information on equipment they have on hand, and their preferred design style. The information entered by the user is entered into an input form using text boxes and selection lists. The entered data is stored in the system's internal memory.

[0699] Input: The user inputs a photo of the room, dimension information, a list of necessary equipment, information on equipment on hand, and design style through the terminal.

[0700] Output: Input data stored in internal memory.

[0701] Specific operation: The user takes a photo of their living room with their smartphone camera and uploads it to the application. They also input the room dimensions (5 meters long x 4 meters wide), select a TV stand and carpet as necessary items, input information about their existing sofa and table, and set their preferred design style to simple modern.

[0702] Step 2:

[0703] The terminal sends the data entered by the user to the server, where the transmission is encrypted via the HTTPS protocol, ensuring reliable communication.

[0704] Input: Input data stored in the internal memory.

[0705] Output: The input data sent to the server.

[0706] Specific behavior: When the user taps the "Submit" button, the entered data is converted to JSON format and sent to the server using the HTTPS protocol.

[0707] Step 3:

[0708] The server receives the data sent from the device, stores it in an analysis database, and uses it in subsequent analysis processes.

[0709] Input: Input data sent from the terminal.

[0710] Output: The input data stored in a database.

[0711] Specific operation: The server parses the received data in JSON format, extracts each item (photo, dimension information, equipment list, design style, etc.), and stores them in a database.

[0712] Step 4:

[0713] The server retrieves photos of the room from the database and performs image analysis using OpenCV, which uses edge detection and object recognition algorithms to identify the contours of the room and the location of furniture.

[0714] Input: A photo of a room stored in a database.

[0715] Output: Room outline and furniture position information (e.g. coordinate data).

[0716] Specific operation: Edges are detected using the OpenCV Canny algorithm, the shape of the furniture is identified using the Hough transform, and the type and location of the furniture are identified using an object recognition algorithm.

[0717] Step 5:

[0718] The server retrieves text data (dimension information, list of required equipment, information on equipment on hand, design style) from the database and performs text data analysis.

[0719] Input: Text data stored in a database.

[0720] Output: Analyzed dimensions, required equipment list, on-hand equipment information, design styles.

[0721] How it works: The server uses NLTK to parse the text data, extract dimensional information as numerical data, obtain the required equipment list and on-hand equipment information, and classify the relevant information based on the design style.

[0722] Step 6:

[0723] The server then integrates the text data and image analysis results and invokes a generative AI model to generate the optimal room layout. This process uses tools such as "DALL·E."

[0724] Input: Analyzed dimensions, required equipment list, equipment on hand, design style, image analysis results (room outline and furniture position information).

[0725] Output: Generated room layout image.

[0726] Specific operation: The server generates a prompt sentence and inputs it into the generative AI model along with the analysis data. The generative AI model then generates the optimal layout image based on this and sends it back to the server. Examples of prompt sentences include:

[0727] User input details:

[0728] Room photo

[0729] Room size: 5 meters by 4 meters

[0730] Required furniture and appliances: TV stand and carpet

[0731] Existing items: Sofa and table

[0732] Preferred style: Simple Modern

[0733] Please generate an optimal room layout image and suggest a list of furniture and appliances.

[0734] Step 7:

[0735] The server creates a list of furnishings to suggest to the user based on the generated layout image, and selects related interior and home appliances based on the analyzed design style.

[0736] Input: The generated room layout image.

[0737] Output: Proposed equipment list.

[0738] Specific behavior: The server adds related items to the list, such as TV stands and carpets that fit the simple modern style. This suggestion list is created based on the results of image and text analysis.

[0739] Step 8:

[0740] The server generates a link to an online shopping site for each proposed item, obtains the product link via API, and provides it to the user.

[0741] Input: Proposed equipment list.

[0742] Output: Purchase links for each fixture.

[0743] Specific operation: The server calls the API of the online shopping site, obtains the purchase link for each suggested item, and adds it to the list.

[0744] Step 9:

[0745] The server sends the final generated room layout image, the proposed equipment list, and a purchase link to the terminal.

[0746] Input: Generated room layout image, proposed fixture list, purchase link.

[0747] Output: The total data sent to the device.

[0748] What it does: The server converts the relevant data into a user-friendly HTML format and sends it to the device, where users can preview the interior and purchase items.

[0749] Step 10:

[0750] The terminal displays to the user an image of the final room layout, a list of suggested furnishings, and a link to purchase.

[0751] Input: Comprehensive data sent from the server.

[0752] Output: A room layout image shown to the user, a suggested furniture list, and a purchase link.

[0753] Specific operation: The user can view the layout image and suggested equipment list through the application on their device, and click on the provided link to go to the shopping site to purchase the items.

[0754] Through the above series of processing steps, the present invention provides a system that allows users to easily and efficiently realize their ideal room and purchase the necessary interior decorations and home appliances.

[0755] (Application example 1)

[0756] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0757] Traditionally, the process of selecting interior decor and home appliances to create the ideal room when moving into a new home has been extremely time-consuming and laborious. It is particularly difficult to visualize the layout of furniture and home appliances and select the appropriate items, and it is even more time-consuming to search for links to purchase each item one by one. It is also not easy to match the user's desired style with existing furniture. There is a need for a system that can solve these problems and efficiently support the creation of the ideal room.

[0758] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0759] In this invention, the server includes input means for inputting a room image, room dimension information, a list of necessary fixtures and electrical equipment, information on existing fixtures and electrical equipment, and a preferred style from the user, transmission means for transmitting data from the input means to the server, analysis means for receiving the data from the transmission means and performing image analysis and text data analysis, generation means for generating a room layout image using a generation AI based on the results of the analysis means, proposal means for generating a list of suggested fixtures and electrical equipment based on the layout generated by the generation means, link generation means for generating links to purchase pages for the fixtures and electrical equipment proposed by the proposal means, display means for transmitting and displaying the data generated by the link generation means to the user, and link acquisition means for acquiring purchase links for the suggested fixtures and electrical equipment. This allows users to easily generate their ideal room layout and purchase the necessary fixtures and electrical equipment.

[0760] A "user" is a user of the system who inputs information for selecting and arranging interior decor and home appliances for a new home.

[0761] "Room footage" refers to photos and videos of the room that users provide to the system.

[0762] "Room dimension information" is information about the physical size of a room, such as the length, width, and height of the room.

[0763] "Fixtures" refers to the interior and furniture of a room, and usually includes chairs, tables, shelves, etc.

[0764] "Electrical equipment" refers to electrical appliances and devices used in a room, such as a television or refrigerator.

[0765] "Input means" refers to a device or interface that allows a user to input room images, dimensional information, a list of fixtures and electrical equipment, etc. into the system.

[0766] "Transmission means" refers to a device or software for transmitting input data to a server.

[0767] "Analysis means" refers to functions and software for performing image analysis and text data analysis based on received data.

[0768] "Generation means" refers to functions and algorithms for generating a room layout image using generation AI based on the results of the analysis means.

[0769] The "proposal means" is a function or software that creates a list of proposed fixtures and electrical equipment based on the generated layout.

[0770] "Link generation means" refers to the functionality or software for generating links to the purchase pages of the proposed fixtures and electrical equipment.

[0771] "Display means" refers to a device or interface for displaying the generated data, suggested lists, and purchase links to the user.

[0772] "Link acquisition means" refers to the functionality or software for acquiring the purchase links for the proposed fixtures and electrical equipment.

[0773] "Style" refers to the interior design or theme that the user prefers, such as simple modern.

[0774] This invention is a system that assists users in purchasing interior decor and home appliances to create their ideal room when moving into a new home. This system uses generative AI to generate a room layout image based on user input and suggests specific fixtures and electrical equipment. The processing flow of the system's program and its specific operation are explained below.

[0775] System Overview

[0776] This system consists of a terminal and a server. The terminal sends the user's input information to the server and displays the final result to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list.

[0777] Program processing flow and operation

[0778] 1. User Input

[0779] The user uses the terminal to input a picture of the room, the room's dimensions, a list of necessary fixtures and electrical equipment, information on the fixtures and electrical equipment they have, and their preferred style.

[0780] For example, you can input a video of the living room of your new home, the dimensions of the room (5m x 4m), the need for a TV stand and carpet, the sofa and table you already have, and your preferred style (simple modern).

[0781] 2. Data Transmission

[0782] The terminal transmits the collected user input data to the server.

[0783] 3. Data Receipt and Storage

[0784] The server receives the data sent from the device and stores it in a database for analysis.

[0785] 4. Image Analysis

[0786] The server retrieves images of the room from the database and performs image analysis, using image recognition algorithms to identify the layout of the room and the location and dimensions of existing furniture. For example, edge detection and object recognition algorithms are used to identify the contours of the room and the location of furniture.

[0787] 5. Text Data Analysis

[0788] The server analyzes the room dimensions, the list of required fixtures and electrical equipment, and the information on existing fixtures and electrical equipment, thereby understanding the size of the room and the required interior specifications.

[0789] 6. Style Analysis

[0790] The server performs an analysis to identify suitable fixtures and electrical equipment based on the user's preferred style (e.g., simple modern).

[0791] 7. Data integration and input information organization

[0792] The server integrates and organizes the results of image analysis, text data analysis, and style analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[0793] 8. Layout Generation Using Generative AI

[0794] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then uses the user's input information and analysis results to automatically arrange the layout and interior.

[0795] 9. Generate a list of fixtures and electrical equipment

[0796] The server creates a list of suggested furniture and electrical equipment for the user based on the room layout image generated by the AI. For example, it adds a simple, modern TV stand or carpet to the list.

[0797] 10. Generate shopping page links

[0798] The server generates a link to a purchasing page for each proposed fixture or electrical device, possibly using an online shopping API.

[0799] 11. Data Transmission

[0800] The server sends the device an image of the final room layout, a list of proposed fixtures and electrical equipment, and a link to purchase each item.

[0801] 12. Displaying Data

[0802] The terminal displays to the user an image of the final room layout, a list of suggested fixtures and electrical equipment, and a link to purchase.

[0803] 13. User Choices and Purchases

[0804] Users can check the information displayed on their device, select the items they like from the suggested fixtures and electrical equipment, and purchase them using the provided purchase link.

[0805] Hardware and software used

[0806] Hardware:

[0807] Smartphone (with camera function)

[0808] software:

[0809] Python, Pillow, and requests

[0810] Examples:

[0811] User A takes a video of their living room with their smartphone and enters the room dimensions (5m x 4m), a list of necessary furniture (TV stand, carpet), a list of furniture they already have (sofa, table), and their preferred style (simple modern) into an input form.

[0812] The app generates the best simple modern layout image and provides purchase links for the "Simple Modern TV Stand" and "Simple Modern Carpet."

[0813] Example prompts to input to a generative AI model:

[0814] Generate the optimal room layout image based on the user's room image, dimensions, required furniture list, existing furniture information, and preferred style.

[0815] Room video: <Video from user>

[0816] Room dimensions: 5m long, 4m wide

[0817] List of necessary furniture: TV stand, carpet

[0818] Current fixtures: sofa, table

[0819] Favorite style: Simple modern

[0820] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0821] Step 1:

[0822] The user uses a terminal to input a video of the room, information about the room's dimensions, a list of necessary fixtures and electrical equipment, information about the fixtures and electrical equipment they currently have, and their preferred style. A smartphone application is used as the input method. The data the user inputs might include, for example, a video of the living room in their new home, the dimensions of the room (5m long x 4m wide), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style being simple and modern. This input information is then saved as input data.

[0823] Step 2:

[0824] The terminal sends the collected user input data to the server. This transmission method uses the HTTP protocol via the Internet. The input data is converted into JSON format and sent to the server. This data transmission procedure passes data related to the user's needs to the server.

[0825] Step 3:

[0826] The server receives the data sent from the device and stores it in a database for analysis. The server, as the receiving means, verifies the accuracy of the data and stores it in the database if there are no problems. The stored data is necessary for subsequent analysis processing and serves as the basis for realizing the user's wishes.

[0827] Step 4:

[0828] The server retrieves images of the room from the database and performs image analysis. Image analysis determines the room layout and the location and dimensions of existing furniture. Specifically, edge detection and object recognition algorithms are used to identify the contours of the room and the location of furniture. The image of the room is used as input data, and data on the layout and dimensions of furniture is output.

[0829] Step 5:

[0830] The server analyzes the room's dimensions, a list of required fixtures and electrical equipment, and information on existing fixtures and electrical equipment. It then analyzes the text data to understand the room's size and required interior specifications. This information is used as input data, and the specific specifications needed for fixture placement and selection are output.

[0831] Step 6:

[0832] The server performs an analysis to identify suitable fixtures and electrical equipment based on the user's preferred style. Analysis of the style data allows for the selection of fixtures based on the user's preferred taste, such as simple modern. Style information is used as input data, and fixtures and electrical equipment that match the user's preferences are output.

[0833] Step 7:

[0834] The server integrates and organizes the results of image analysis, text data analysis, and style analysis. This clarifies the design conditions that are optimal for the user's wishes and the characteristics of the room. Multiple analysis results are integrated and the optimal layout conditions are output.

[0835] Step 8:

[0836] The server calls the generation AI based on the integrated data and generates an optimal room layout image. The generation AI uses the user's input information and analysis results to automatically arrange the layout and interior. The following text is used as an input prompt:

[0837] Generate the optimal room layout image based on the user's room image, dimensions, required furniture list, existing furniture information, and preferred style.

[0838] Room video: <Video from user>

[0839] Room dimensions: 5m long, 4m wide

[0840] List of necessary furniture: TV stand, carpet

[0841] Current fixtures: sofa, table

[0842] Favorite style: Simple modern

[0843] Step 9:

[0844] The server creates a list of furniture and electrical equipment to suggest to the user based on the room layout image generated by the AI. A list of appropriate furniture and electrical equipment is output from the generated layout image, allowing the user to select the optimal interior.

[0845] Step 10:

[0846] The server generates a link to the purchase page for each proposed fixture or electrical device. The link generation method uses an online shopping API, which outputs a link that allows the user to directly access the purchase page.

[0847] Step 11:

[0848] The server sends the final room layout image, a list of proposed fixtures and electrical equipment, and a link to purchase each product to the terminal. The data is sent via the HTTP protocol over the Internet. The data output contains all the necessary information for the user.

[0849] Step 12:

[0850] The terminal displays the final room layout image, a list of suggested fixtures and electrical equipment, and a link to purchase. The user can review this information and evaluate the room layout. Visual information is output for optimal interior selection.

[0851] Step 13:

[0852] Users can check the information displayed on their devices, select the items they like from the suggested fixtures and electrical equipment, and purchase them using the provided purchase link. This simplifies the selection and purchase process, providing users with concrete steps to create their ideal room.

[0853] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0854] The present invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room for their new home. By incorporating an emotion engine that recognizes the user's emotions, the system can provide more appropriate suggestions based on the user's emotional state. The processing flow and specific operations of the system's program are explained below.

[0855] System Overview

[0856] This system consists of a terminal and a server. The terminal sends the user's input information to the server, analyzes the user's emotions through an emotion engine, and displays the final results to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list, adjusting them based on the user's emotional state.

[0857] Program processing flow and operation

[0858] 1. User Input

[0859] The user uses a terminal to input a photo of the room, information about the room's size, a list of the furniture and appliances they need, information about the furniture and appliances they already have, and their preferred style.The emotion engine then analyzes the user's voice, facial expression, and input speed in real time to grasp the user's emotional state.For example, a user might input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the fact that they already have a sofa and table, and their preferred style (simple modern).

[0860] 2. Data Transmission

[0861] The terminal transmits the data input by the user and the emotion data analyzed by the emotion engine to the server.

[0862] 3. Data Receipt and Storage

[0863] The server receives the data sent from the device and stores it in a database for analysis.

[0864] 4. Image Analysis

[0865] The server retrieves the room's photo data from the database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. For example, edge detection algorithms and object recognition techniques are used to detect the contours of the room and the location of furniture.

[0866] 5. Text Data Analysis

[0867] The server analyzes the text data provided by the user, including the room size, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, to understand the room size and required interior specifications.

[0868] 6. Taste and Emotion Analysis

[0869] The server performs analysis to identify suitable furniture and appliances based on the user's preferred taste (e.g., simple modern). It also adjusts the taste and style of the furniture and appliances it recommends based on the user's emotional state received from the emotion engine. For example, if the user is feeling stressed, it will suggest interior items that will help them relax.

[0870] 7. Data integration and input information organization

[0871] The server integrates and organizes the results of image analysis, text data analysis, taste analysis, and emotion analysis from the emotion engine, thereby clarifying the optimal design conditions for the user's wishes and the characteristics of the room.

[0872] 8. Layout Generation Using Generative AI

[0873] The server then calls a generative AI based on the integrated data to generate an optimal room layout image. This generative AI automatically arranges furniture and appliances to create the optimal layout based on the user's input data, analysis results, and emotional state.

[0874] 9. Generate a list of interior, furniture, and home appliances

[0875] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a TV stand or carpet with a simple, modern design to the list. The server also takes into account the user's emotional state to further optimize the options.

[0876] 10. Generate shopping page links

[0877] The server generates a link to the purchase page for each suggested piece of furniture or home appliance, taking into account the size and placement information of each product. Based on the analysis results of the emotion engine, the server makes suggestions that will increase user satisfaction.

[0878] 11. Data Transmission

[0879] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[0880] 12. Displaying Data

[0881] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[0882] 13. User Choices and Purchases

[0883] Users can select the furniture or appliances they like from the suggested list displayed on their device and complete the purchase using the provided link. At this time, they can also check the specific interior layout while making their selection.

[0884] This concludes the description of the embodiment of the invention. This system not only allows users to efficiently and effectively select and arrange furniture and home appliances that suit their room and preferences, but also provides links to purchase each item. Furthermore, by adjusting the suggestions based on the user's emotional state, it is possible to create a room that is more satisfying.

[0885] The processing flow will be explained below.

[0886] Step 1:

[0887] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they currently have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style (simple modern).

[0888] Step 2:

[0889] The device recognizes the user's emotional state through the user's tone of voice, facial expression, and input speed. The emotion engine analyzes this information and estimates the user's emotional state, for example, determining whether the user is relaxed or nervous.

[0890] Step 3:

[0891] The terminal transmits all data input by the user and emotion data analyzed by the emotion engine to the server.

[0892] Step 4:

[0893] The server receives the data sent from the device and stores it in a database for analysis.

[0894] Step 5:

[0895] The server retrieves the room's photo data from the database and performs image analysis, using edge detection algorithms and object recognition techniques to identify the room layout and the location and dimensions of existing furniture. For example, it recognizes the boundaries between the floor and walls of the room and identifies the placement of existing furniture.

[0896] Step 6:

[0897] The server analyzes the room size information, the list of necessary furniture and appliances, and the information on the furniture and appliances currently available as text data. This clarifies the specific dimensions of the room and the interior requirements. For example, it determines the size of the TV stand and carpet that are appropriate for the living room.

[0898] Step 7:

[0899] The server analyzes the user's preferred taste (e.g., simple modern) and identifies suitable furniture and home appliances. It also adjusts the taste and style of the furniture and home appliances it recommends based on the emotional data obtained from the emotion engine. For example, if the user wants to relax, it will suggest furniture in soft colors.

[0900] Step 8:

[0901] The server integrates the results of image analysis, text data analysis, taste analysis, and emotion analysis from the emotion engine to determine optimal design conditions.

[0902] Step 9:

[0903] The server uses the integrated data to call a generative AI to generate an optimal room layout image. This generative AI automatically arranges furniture and appliances based on the user's input data and analysis results, and creates the optimal layout by taking emotional data into consideration.

[0904] Step 10:

[0905] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a TV stand or carpet with a simple, modern design to the list, reflecting the results of sentiment analysis.

[0906] Step 11:

[0907] The server generates links to purchase pages for the suggested furniture and appliances, taking into account the size and placement of each product and adjusting them according to the user's emotional state.

[0908] Step 12:

[0909] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[0910] Step 13:

[0911] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[0912] Step 14:

[0913] Users can select their favorite furniture or appliances from the suggested list displayed on their device and purchase them using the provided link, or they can rearrange the proposed interior. Here, they can make their final selection while checking the specific interior layout.

[0914] Example 2

[0915] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0916] Conventional purchasing support systems for room interiors and home appliances require users to manually sort through large amounts of information and determine the optimal interior layout, which is extremely time-consuming. Furthermore, since suggestions are not based on the user's emotional state, this can result in low user satisfaction. There is a need for a system that overcomes these drawbacks, reduces the user's time and effort, and provides suggestions that are in line with individual emotions.

[0917] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0918] In this invention, the server includes: input means for inputting image data of the room, room dimension information, a list of necessary items, information on items on hand, and a preferred interior style from the user; communication means for transmitting the data from the input means to a central processing unit; analysis means for receiving the data from the communication means and performing image analysis and text data analysis; emotion recognition means for grasping the user's emotional state; generation means for generating a room layout image using a generative AI model based on the results of the analysis means and the emotion recognition means; suggestion means for generating a list of suggested items based on the layout generated by the generation means; link generation means for generating links to purchase pages for the items suggested by the suggestion means; and display means for transmitting the data generated by the link generation means to the user and displaying it. This allows the user to receive suggestions for optimal interior layout and items based on their emotional state.

[0919] "User" refers to a person who uses this system to receive suggestions for room interiors and home appliances.

[0920] "Input means" refers to a device or software that allows a user to input image data of a room, dimensional information, a list of necessary items, information on items on hand, and a preferred interior style.

[0921] "Communication means" refers to a method or device for transmitting data from an input means to a central processing unit.

[0922] "Analysis means" refers to devices and algorithms that perform image analysis and text data analysis based on received data.

[0923] "Emotion recognition means" refers to devices or software that analyze a user's voice, facial expressions, input speed, etc. to understand their emotional state.

[0924] "Generation means" refers to a device or software for generating a room layout image using a generative AI model based on the results of the analysis means and emotion recognition means.

[0925] The "suggestion means" refers to a method or apparatus for creating a suggested item list based on the layout generated by the generation means.

[0926] The "link generating means" refers to a method or device for generating a link to a purchase page for the item suggested by the suggesting means.

[0927] The "display means" refers to a device or software for transmitting the data generated by the link generation means to the user and visually presenting it to the user.

[0928] A "generative AI model" refers to an artificial intelligence algorithm that automatically creates the optimal room layout based on user input data and analysis results.

[0929] The present invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room when moving into a new home. This system is configured using a terminal and a server. The detailed configuration and operation of the system are described below.

[0930] overview

[0931] This system assists users in the process of selecting the best interior and home appliances for their new home. The system incorporates an emotion engine to constantly grasp the user's emotional state and make more appropriate suggestions. Users can input data using their device to review and purchase the suggested interior and home appliances.

[0932] Specific actions

[0933] 1. User Input

[0934] The user uses the device to input image data of the room they will be living in, the room's dimensions, a list of items they need, information about items they already have, and their preferred interior style. The emotion engine analyzes the user's voice, facial expressions, and input speed in real time to grasp their emotional state.

[0935] Example: Take a photo of the living room of your new home with your smartphone and upload it. Then, enter the room dimensions (5m x 4m), items you need (TV stand, carpet), furniture you already have (sofa, table), and your preferred interior style (simple modern) in the input fields.

[0936] 2. Data Transmission

[0937] The terminal transmits data input by the user and emotion data analyzed by the emotion engine to the server. This data includes images, text, and emotion analysis results.

[0938] 3. Data Receipt and Storage

[0939] The server receives the data sent from the device and stores it in a database for analysis. The stored data includes photos of the room, text data, and emotion analysis data.

[0940] 4. Image Analysis

[0941] The server retrieves photos of the room from a database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. Techniques used include edge detection and object recognition algorithms.

[0942] Example: The server uses the image analysis library OpenCV to extract the contours of a room and the position of furniture using an edge detection algorithm.

[0943] 5. Text Data Analysis

[0944] The server performs text analysis of the room dimension information, list of required items, and information on items on hand provided by the user to understand the size of the room and the specifications of the required items.

[0945] 6. Taste and Emotion Analysis

[0946] The server performs analysis to identify suitable items based on the user's preferred interior style (e.g., simple modern), and adjusts the style of the suggested items based on the user's emotional state received from the emotion engine.

[0947] Example: If the user is feeling stressed, suggest interior design that will help them relax.

[0948] 7. Data integration and input information organization

[0949] The server integrates and organizes the image analysis results, text data analysis results, taste analysis results, and emotion analysis results from the emotion engine.

[0950] 8. Layout Generation Using Generative AI Models

[0951] The server calls up a generative AI model based on the integrated data and generates an optimal room layout image.

[0952] Example prompt for a generative AI model:

[0953] "Generate a room layout in a simple, modern interior style using the user's input data: room dimensions (5m x 4m), items on hand (sofa and table), and a list of items needed (TV stand and carpet). The user's emotional state should be relaxed."

[0954] 9. Generate a list of interior items

[0955] The server creates a list of items to suggest to the user based on the room layout image generated by the generative AI model.

[0956] 10. Generate shopping page links

[0957] The server generates a link to the purchase page for each suggested item, taking into account the analysis results of the emotion engine to ensure a high level of user satisfaction.

[0958] 11. Data Transmission

[0959] The server sends the generated final data (room layout image, suggested item list, and purchase link) to the terminal.

[0960] 12. Displaying Data

[0961] The terminal displays the final room layout image, a list of suggested items, and a link to purchase them to the user, allowing the user to select the interior design while reviewing them.

[0962] 13. User Choices and Purchases

[0963] Users can select the items they like from the suggested list displayed on their device and complete the purchase using the provided link, while also checking the specific interior layout.

[0964] This system not only allows users to efficiently and effectively select and arrange interior and home appliances that suit their room and preferences, but also provides links to purchase each item. Furthermore, by adjusting the suggestions based on the user's emotional state, it is possible to create a room that is more satisfying.

[0965] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0966] Step 1:

[0967] User Input

[0968] The user uses the device to input image data of the room, room dimensions, a list of items needed, information about items they have on hand, and their preferred interior style. The emotion engine analyzes voice, facial expressions, and input speed in real time to grasp the user's emotional state.

[0969] Input: room image data, room dimensions, list of necessary items, information on items on hand, preferred interior style

[0970] Output: User input data and emotion data stored on the device

[0971] How it works: The user takes a photo of the living room of their new home with their smartphone and uploads it to the application. Then, they fill in the input fields with the room dimensions (5m x 4m), items they need (TV stand, carpet), furniture they already have (sofa, table), and their preferred interior style (simple modern).

[0972] Step 2:

[0973] Data transmission

[0974] The terminal transmits data input by the user and emotion data analyzed by the emotion engine to the server.

[0975] Input: User input data and emotion data stored on the device

[0976] Output: User input data and emotion data sent to the server

[0977] Specific operation: The terminal uses the API to send input data in JSON format to the server.

[0978] Step 3:

[0979] Data reception and storage

[0980] The server receives the data sent from the device and stores it in a database for analysis.

[0981] Input: User input data and emotion data sent to the server

[0982] Output: User data and emotion data stored in a database

[0983] Specific operation: The server calls the database save function to save the received data. For example, the image data of the room is saved in Blob format, and the text data is saved in a structured database.

[0984] Step 4:

[0985] Image analysis

[0986] The server retrieves image data of the room from the database and performs image analysis to determine the room layout, the location of existing furniture, and its dimensions.

[0987] Input: Image data of the room stored in the database

[0988] Output: Room layout, existing furniture locations, dimensions

[0989] Specific operation: The server uses the OpenCV library to extract the room contours and furniture positions using an edge detection algorithm.

[0990] Step 5:

[0991] Text Data Analysis

[0992] The server performs text analysis of the room dimension information, list of required items, and information on items on hand provided by the user to understand the size of the room and the specifications of the required items.

[0993] Input: Room dimensions stored in the database, list of required items, and information on items on hand

[0994] Output: Analyzed room size, required item specifications

[0995] What it does: The server uses natural language processing (NLP) tools to parse the text input data and get the room dimensions and a list of required items.

[0996] Step 6:

[0997] Taste and emotion analysis

[0998] The server performs analysis to identify suitable items based on the user's preferred interior style (e.g., simple modern), and adjusts the style of the suggested items based on the user's emotional state received from the emotion engine.

[0999] Input: User's preferred interior style, sentiment analysis results

[1000] Output: Adjusted product list and style suggestions

[1001] How it works: The server uses a taste analysis algorithm to search for furniture that suits a simple modern style, and customizes suggestions based on the results of sentiment analysis.

[1002] Step 7:

[1003] Data integration and input organization

[1004] The server integrates and organizes the results of image analysis, text data analysis, taste analysis, and emotion analysis.

[1005] Input: Image analysis results, text data analysis results, taste analysis results, emotion analysis results

[1006] Output: Integrated interior proposal data

[1007] Specific operation: The server integrates these analysis results into a single data structure, compiling all the information necessary to propose the optimal interior design.

[1008] Step 8:

[1009] Layout generation using generative AI models

[1010] The server calls up a generative AI model based on the integrated data and generates an optimal room layout image.

[1011] Input: Integrated interior proposal data

[1012] Output: Generated room layout image

[1013] Specific operation: The server sends a prompt to the generative AI model (e.g., "Generate a simple, modern living room layout based on the user's input data") and retrieves the returned results.

[1014] Step 9:

[1015] Generate a list of interior items

[1016] The server creates a list of items to suggest to the user based on the room layout image generated by the generative AI model.

[1017] Input: Generated room layout image

[1018] Output: List of suggested items

[1019] How it works: The server checks a database of furniture and appliances to create a list of items that match the layout image.

[1020] Step 10:

[1021] Generate shopping page links

[1022] The server generates a link to a purchase page for each suggested item, taking into account the analysis results of the emotion engine to maximize user satisfaction.

[1023] Input: List of suggested items, sentiment analysis results

[1024] Output: Link to purchase page

[1025] What it does: The server collects online shop links for each item and adds the links to a list for easy user access.

[1026] Step 11:

[1027] Sending data

[1028] The server sends the generated final data (room layout image, suggested item list, and purchase link) to the terminal.

[1029] Input: Final data (room layout image, suggested item list, purchase link)

[1030] Output: Final data sent to the terminal

[1031] Specific operation: The server uses the API to send the final data to the terminal.

[1032] Step 12:

[1033] Viewing Data

[1034] The terminal displays the final room layout image, a list of suggested items, and a link to purchase them to the user, allowing the user to select the interior while reviewing them.

[1035] Input: Final data sent from the server

[1036] Output: Layout image displayed to the user, item list, purchase link

[1037] Specific operation: The device displays the acquired data on the application's UI, making it easy for the user to view.

[1038] Step 13:

[1039] User Choice and Purchase

[1040] The user selects the item they like from a list of suggestions displayed on their device and completes the purchase using the provided link.

[1041] Input: Displayed suggestion list and purchase link

[1042] Output: Purchased item

[1043] Specific operation: The user clicks on the purchase link, goes to the online shop page and purchases the product.

[1044] This allows users to efficiently select and arrange interior and home appliances that suit their room and preferences, and receive suggestions based on their emotional state.

[1045] (Application example 2)

[1046] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1047] Conventional interior design suggestion systems have difficulty in making suggestions that take into account the user's emotional state, which results in a failure to increase user satisfaction. Furthermore, when purchasing in a physical store, they are unable to make optimal suggestions based on real-time interior layout or emotions, which means it takes a lot of time and effort for users to realize their ideal room.

[1048] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1049] In this invention, the server includes: emotion analysis means for analyzing the user's emotional state and adjusting the proposal content based on that state; input means for inputting from the user an image of the room, room dimension information, a list of necessary household goods and home appliances, information on household goods and home appliances currently in use, and a preferred design; transmission means for transmitting data from the input means to a central processing unit; analysis means for receiving the data from the transmission means and performing image analysis and text data analysis; generation means for generating a room layout image using a generation AI based on the results of the analysis means; proposal means for generating a list of proposed household goods and home appliances based on the layout generated by the generation means; link generation means for generating links to purchasing pages for the household goods and home appliances proposed by the proposal means; and display means for transmitting and displaying the data generated by the link generation means to the user. This enables optimal interior proposals in real time while taking the user's emotional state into consideration, and facilitates smooth purchases in physical stores.

[1050] "User" refers to a person who uses the system.

[1051] "Room image" refers to a photo of a room taken by a user.

[1052] "Room dimension information" refers to information about the length, width, and height of a room.

[1053] "List of necessary household goods and home appliances" refers to a list of household goods and home appliances that the user wants to purchase.

[1054] "Information about existing household goods and home appliances" refers to information about household goods and home appliances that the user already owns.

[1055] "Preferred design" refers to the interior style or theme that the user prefers.

[1056] "Input means" refers to a device or method by which a user inputs information into a system.

[1057] The "transmission means" refers to a function for transmitting data acquired from the input means to the central processing unit.

[1058] "Analysis means" refers to a device or method that analyzes received data and performs image and text data analysis.

[1059] "Generation means" refers to the function of generating a room layout image using generation AI based on the results of the analysis means.

[1060] The "suggestion means" refers to a function for generating a list of suggested household goods and home appliances based on the arrangement generated by the generation means.

[1061] The "link generating means" refers to a function for generating a link to a purchase page for the household goods / home appliances suggested by the suggesting means.

[1062] The "display means" refers to a function that displays the data generated by the link generation means to the user.

[1063] "Emotion analysis means" refers to a function that analyzes the user's emotional state and adjusts the content of suggestions based on that state.

[1064] "Central Processing Unit" refers to the main unit for processing, analyzing and generating user input data.

[1065] The present invention is a system that supports the purchase of interior decorations and home appliances to realize the ideal room when a user moves into a new home. By combining this system with emotion analysis means, it is possible to make more appropriate suggestions according to the user's emotional state.

[1066] System configuration

[1067] This system is mainly composed of terminals and servers, and is explained in detail below.

[1068] Terminal

[1069] The terminal is a device that inputs information through user operation and sends it to the server. The terminal is equipped with hardware such as a camera, microphone, and display, and performs the following functions:

[1070] Input means: The user uses the terminal to input an image of the room, room dimensions, a list of necessary household goods and appliances, information about the household goods and appliances they already have, and their preferred design.

[1071] Emotion analysis means: When a user inputs information, voice, facial expressions, input speed, etc. are analyzed in real time to understand the user's emotional state.

[1072] Transmission method: The input data and the emotion analysis results are sent to the server.

[1073] server

[1074] The server receives data sent from the terminal, analyzes it, generates it, and makes suggestions. It has the following functions:

[1075] Analysis methods: Image analysis and text data analysis are performed. Image analysis uses edge detection algorithms and object recognition techniques to identify the room layout and the location of existing furniture. Text data analysis analyzes the room dimensions and specifications of required furniture.

[1076] Generation method: Based on the analysis results, a generative AI is used to generate an optimal room layout image.

[1077] Proposal method: Based on the generated layout image, a list of suggested household goods and home appliances is generated.

[1078] Link generation method: Generate a link to the purchase page for the suggested household goods and appliances.

[1079] Display means: The generated data is sent to the terminal and displayed to the user.

[1080] Detailed System Operation

[1081] Program processing

[1082] The server uses image processing libraries such as OpenCV to analyze the room photos sent by the user. It also uses a natural language processing engine to analyze text data. It uses a generative AI model (such as GAN or VAE) to generate an optimal room layout image based on the analysis results.

[1083] Emotion analysis means

[1084] The emotion analysis method uses an emotion recognition engine (such as Microsoft Azure's Emotion API or Google Cloud's Vision API) to analyze the user's emotional state in real time from their voice and facial expressions. This allows the system to suggest interior designs that will help the user relax if they are feeling stressed, or that will help them maintain their excitement if they are enjoying themselves.

[1085] Specific examples

[1086] For example, suppose a user takes a photo of the living room of their new home and inputs the room dimensions (5m x 4m), the furniture they need (TV stand and carpet), the furniture they already have (sofa and table), and their preferred design (simple modern). Based on this information and the results of sentiment analysis, the server uses a generative AI model to generate an optimal layout image. It then creates a list of suggested furniture and appliances and generates a link to the purchasing page to display it to the user.

[1087] Prompt Sentence Examples

[1088] "Generate an optimal room layout image based on a photo of the room, size information, a list of necessary furniture and appliances, information on the furniture and appliances currently in use, and the user's preferred design (e.g., simple modern). Please also include suggestions based on the user's emotional state (e.g., relaxation)."

[1089] This system not only allows users to efficiently and effectively select and arrange furniture and appliances that suit their room and preferences, but also provides links to purchase each item. Furthermore, by adjusting the suggestions based on the user's emotional state, it is possible to create a room that is more satisfying.

[1090] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1091] Step 1:

[1092] The user uses the terminal to input a picture of the room, dimension information, a list of necessary household goods and appliances, information about the household goods and appliances they already have, and their preferred design.

[1093] Input: Room photo, dimensions, list of necessary furniture and appliances, information on existing furniture and appliances, preferred design

[1094] Output: Input data (a series of information)

[1095] Specific operation: The user uses the application on their smartphone or tablet to take a photo of the room using the camera function and fill out an input form with various information. This information is temporarily stored on the device as integrated data.

[1096] Step 2:

[1097] The terminal uses an emotion analysis means to analyze the user's emotional state in real time from their voice and facial expressions.

[1098] Input: User's voice, facial expressions

[1099] Output: Sentiment analysis data

[1100] How it works: The device uses a microphone and camera to capture the user's voice and facial expressions. This data is then fed into an emotion analysis engine (e.g., Emotion API) to analyze the user's emotional state (e.g., stress, joy) in real time.

[1101] Step 3:

[1102] The terminal transmits the input data and the emotion analysis results to the server.

[1103] Input: Input data, sentiment analysis data

[1104] Output: Data sent to the server

[1105] Specific operation: The device combines the user's input data and emotion analysis data and sends it to the server as a single data packet. The communication protocol is HTTPS or similar, ensuring secure transmission.

[1106] Step 4:

[1107] The server stores the received data in a database for analysis and performs image analysis and text data analysis.

[1108] Input: Data sent to the server

[1109] Output: Image analysis results, text data analysis results

[1110] Specific operation: The server stores the received data in an analysis database. It then analyzes the room photo using image processing libraries such as OpenCV, and performs edge detection algorithms and object recognition. At the same time, it uses a text data analysis engine to analyze the dimensions and furniture list.

[1111] Step 5:

[1112] Based on the analysis results, the server uses a generative AI model to generate an optimal room layout image.

[1113] Input: Image analysis results, text data analysis results, emotion analysis data

[1114] Output: Generated aligned image

[1115] How it works: The server takes the analysis results and emotion data as input and provides them to a generative AI model (e.g., GAN), which then generates an optimal room layout image that matches the user's preferences.

[1116] Step 6:

[1117] The server generates a list of suggested household goods and home appliances based on the generated layout image.

[1118] Input: Generated aligned image

[1119] Output: List of suggested household goods and appliances

[1120] How it works: The server analyzes the layout images output by the generative AI model and automatically generates a list of optimal proposals, including detailed information such as the size and design of furniture and appliances.

[1121] Step 7:

[1122] The server generates a link to the purchase page for the suggested household goods and home appliances.

[1123] Input: List of suggested household goods and appliances

[1124] Output: Link to purchase page

[1125] How it works: For each item in the suggestion list, the server searches an online shopping database and generates a link to the page where it can be purchased, optimized to take into account the user's emotional state.

[1126] Step 8:

[1127] The server transmits the generated data to the terminal, which displays it to the user.

[1128] Input: Purchase link, proposal list, placement image

[1129] Output: What is displayed to the user

[1130] Specific operation: The server sends a link to the purchase page, a list of suggestions, and layout images to the device. The device receives this data and displays it in a format that is easy for the user to view. The user can then select and purchase interior items based on the displayed information.

[1131] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1132] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1133] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1134] [Third embodiment]

[1135] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1136] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1137] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1138] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1139] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1140] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1141] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1142] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1143] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1144] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1145] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1146] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1147] This invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room when moving into a new home. This system uses a generative AI to generate a room layout image based on user input and suggests specific interior decor and home appliances. The processing flow of the system's program and its specific operation are explained below.

[1148] System Overview

[1149] This system consists of a terminal and a server. The terminal sends the user's input information to the server and displays the final result to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list.

[1150] Program processing flow and operation

[1151] 1. User Input

[1152] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they currently have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style (simple modern).

[1153] 2. Data Transmission

[1154] The terminal transmits the collected user input data to the server.

[1155] 3. Data Receipt and Storage

[1156] The server receives the data sent from the device and stores it in a database for analysis.

[1157] 4. Image Analysis

[1158] The server retrieves a photo of the room from the database and performs image analysis, using image recognition algorithms to identify the room's layout and the location and dimensions of existing furniture, for example, using edge detection and object recognition algorithms to identify the contours of the room and the location of furniture.

[1159] 5. Text Data Analysis

[1160] The server analyzes the room size information, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, thereby understanding the size of the room and the required interior specifications.

[1161] 6. Taste Analysis

[1162] The server performs an analysis to identify suitable furniture and appliances based on the user's preferred style (e.g., simple modern).

[1163] 7. Data integration and input information organization

[1164] The server integrates and organizes the results of image analysis, text data analysis, and taste analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[1165] 8. Layout Generation Using Generative AI

[1166] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then uses the user's input information and analysis results to automatically arrange the layout and interior.

[1167] 9. Generate a list of interior, furniture, and home appliances

[1168] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a simple, modern TV stand or carpet to the list.

[1169] 10. Generate shopping page links

[1170] The server generates a link to a purchasing page for each suggested piece of furniture or appliance, possibly using an online shopping API.

[1171] 11. Data Transmission

[1172] The server sends the device an image of the final room layout, a list of suggested furniture and appliances, and a link to purchase each item.

[1173] 12. Displaying Data

[1174] The device displays to the user an image of the final room layout, a list of suggested furniture and appliances, and a link to purchase.

[1175] 13. User Choices and Purchases

[1176] Users can check the information displayed on their device, select the furniture or home appliances they like from the suggested items, and purchase them using the provided purchase link.

[1177] This concludes the description of the embodiment of the invention. This system not only allows users to efficiently and effectively select and arrange furniture and home appliances that suit their room and preferences, but also provides links to purchase each item at the same time. This allows users to easily create their ideal room.

[1178] The processing flow will be explained below.

[1179] Step 1:

[1180] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they already have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), that they need a TV stand and carpet, that they already have a sofa and table, and that their preferred style is simple and modern.

[1181] Step 2:

[1182] The terminal collectively transmits the data input by the user to the server.

[1183] Step 3:

[1184] The server receives the data sent from the device and stores it in a database for analysis.

[1185] Step 4:

[1186] The server retrieves the room's photo data from the database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. For example, edge detection algorithms and object recognition techniques are used to detect the contours of the room and the location of furniture.

[1187] Step 5:

[1188] The server analyzes the text data provided by the user, including the room size, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, to understand the room size and required interior specifications.

[1189] Step 6:

[1190] The server performs analysis to identify suitable furniture and appliances based on the user's preferred style (e.g., simple modern).

[1191] Step 7:

[1192] The server integrates and organizes the results of image analysis, text data analysis, and taste analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[1193] Step 8:

[1194] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then automatically arranges furniture and appliances based on the user's input data and analysis results to create the optimal layout.

[1195] Step 9:

[1196] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the AI. For example, it adds a TV stand or carpet with a simple, modern design to the list.

[1197] Step 10:

[1198] The server generates a link to the online shopping site for each proposed piece of furniture or appliance, taking into account the size and placement information of each item.

[1199] Step 11:

[1200] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[1201] Step 12:

[1202] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[1203] Step 13:

[1204] Users can select the furniture or appliances they like from the suggested list displayed on their device and complete the purchase using the provided link. At this time, they can also check the specific interior layout while making their selection.

[1205] Example 1

[1206] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1207] In conventional interior and home appliance purchasing support systems, it has been difficult for users to effectively plan the layout, interior, and placement of home appliances in a room. In particular, there was a lack of a way to generate an optimal layout while taking into account the user's preferences, existing furniture, and room size. In addition, generating purchase links had to be done manually, which was a cumbersome and time-consuming task for users. These challenges made it difficult to efficiently create the ideal room.

[1208] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1209] In this invention, the server includes input means for allowing a user to input a photo of the room, room dimension information, a list of necessary fixtures, information about fixtures on hand, and a preferred design style, transmission means for transmitting data from the input means to the server, analysis means for receiving the data from the transmission means and performing image analysis and text data analysis, generation means for generating a room layout image using a generative AI model based on the results of the analysis means, proposal means for generating a proposed fixture list based on the layout generated by the generation means, link generation means for generating links to purchasing pages for the fixtures proposed by the proposal means, and display means for transmitting and displaying the data generated by the link generation means to the user. This enables users to efficiently design a room with an ideal layout based on their preferences and easily purchase the necessary fixtures.

[1210] A "user" is an individual or organization that uses the system to receive support for room layout, interior design, and purchasing home appliances.

[1211] A "terminal" is a device used by a user to provide input information to the system, and includes a PC, smartphone, tablet, etc.

[1212] The "server" is a central processing unit that receives data sent from users, analyzes it, and generates layout images and proposal lists.

[1213] The "input means" is an interface that allows the user to input a photo of the room, dimension information, a list of necessary equipment, information on equipment that is on hand, and a preferred design style.

[1214] The "transmission means" is a means for transmitting the user's input data from the terminal to the server.

[1215] The "analysis means" is a means for analyzing received data in the server and performing image analysis and text data analysis.

[1216] The "generation means" is a means for generating a room layout image using a generative AI model based on the results of the analysis means.

[1217] The "proposing means" is a means for generating an equipment list to be proposed to the user based on the layout generated by the generating means.

[1218] The "link generating means" is a means for generating a link to a purchasing page for the equipment suggested by the suggesting means.

[1219] The "display means" is a means for transmitting the data generated by the link generation means to the user and displaying it.

[1220] "Image analysis" is a technology that processes photos of a room sent by a user and identifies the outline of the room and the location of furniture.

[1221] "Text data analysis" is a technology that analyzes the room dimension information, required equipment list, equipment information on hand, and preferred design style sent by the user.

[1222] A "generative AI model" is an artificial intelligence model that generates optimal room layout images based on user input information and analysis results.

[1223] A "prompt sentence" is an input sentence used to give instructions to a generative AI model.

[1224] "Fixtures" is a general term for decorative and functional items, including interior furnishings and home appliances.

[1225] "Design style" is a concept that refers to a particular theme or aesthetic of room decoration or interior that a user prefers.

[1226] This invention is a system that efficiently supports the purchase of interior decor and home appliances to create an ideal room when a user moves into a new residence. This system is composed of a terminal and a server. The terminal sends the user's input information to the server, and the server uses a generative AI model to generate a room layout image and a proposal list. The final data is then provided to the user.

[1227] Input Method

[1228] Users use their device to input a photo of their room, its dimensions, a list of necessary furnishings, information about the furnishings they already have, and their preferred design style. For example, a user can take a photo of their living room with their smartphone and upload it to the application. They can also input the room dimensions (5 meters long and 4 meters wide), select a TV stand and carpet as necessary items, input the sofa and table they already have, and set their preferred design style to simple modern.

[1229] Transmission method

[1230] The terminal sends the user's input data collected through the input means to the server. This transmission is securely performed using the HTTPS protocol. The input data is converted to JSON format, encrypted, and sent to the server.

[1231] Analysis means

[1232] The server receives data sent from the device and stores it in a database for analysis. For image analysis, the open-source image processing library "OpenCV" is used, using edge detection and object recognition algorithms to identify the outline of the room and the location of furniture. For text data analysis, the "NLTK" library is used to analyze room dimensions and a list of necessary equipment. For taste analysis, generative AI models such as "GPT-3" are used.

[1233] generation means

[1234] Based on the analysis results, the server uses a generative AI model to generate a room layout image. Specifically, it uses a generative AI model such as "DALL·E" and generates the optimal room layout image by inputting a text prompt. The following text is an example of a prompt that can be used:

[1235] Example prompt sentence:

[1236] User input details:

[1237] Room photo

[1238] Room size: 5 meters by 4 meters

[1239] Required furniture and appliances: TV stand and carpet

[1240] Existing items: Sofa and table

[1241] Preferred style: Simple Modern

[1242] Please generate an optimal room layout image and suggest a list of furniture and appliances.

[1243] Proposal means

[1244] The server then creates a list of suggested furnishings for the user based on the generated layout image. For example, it adds a TV stand and carpet suitable for a simple modern style to the list. This suggestion is based on the analyzed user preferences.

[1245] Link Generation Methods

[1246] For each proposed item, the server generates a link to an online shopping site to purchase the item. Specifically, it uses the API of a popular online marketplace to retrieve the product link.

[1247] Display means

[1248] The server sends the final room layout image, the proposed equipment list, and a purchase link for each item to the terminal, which then displays this information to the user. For example, the user can view the layout image and the proposed equipment list through the application interface and click the purchase link to go to a shopping site.

[1249] User Choice and Purchase

[1250] Users can select their favorite items based on the suggested information and complete the purchase process using the provided purchase link, allowing them to efficiently create their ideal room.

[1251] The system of the present invention allows users to simultaneously obtain layout suggestions that suit their room and preferences, a list of necessary fixtures, and links to purchase each item, making it easy for users to create their ideal room.

[1252] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1253] Step 1:

[1254] The user uses the terminal to input a photo of the room, room dimensions, a list of necessary equipment, information on equipment they have on hand, and their preferred design style. The information entered by the user is entered into an input form using text boxes and selection lists. The entered data is stored in the system's internal memory.

[1255] Input: The user inputs a photo of the room, dimension information, a list of necessary equipment, information on equipment on hand, and design style through the terminal.

[1256] Output: Input data stored in internal memory.

[1257] Specific operation: The user takes a photo of their living room with their smartphone camera and uploads it to the application. They also input the room dimensions (5 meters long x 4 meters wide), select a TV stand and carpet as necessary items, input information about their existing sofa and table, and set their preferred design style to simple modern.

[1258] Step 2:

[1259] The terminal sends the data entered by the user to the server, where the transmission is encrypted via the HTTPS protocol, ensuring reliable communication.

[1260] Input: Input data stored in the internal memory.

[1261] Output: The input data sent to the server.

[1262] Specific behavior: When the user taps the "Submit" button, the entered data is converted to JSON format and sent to the server using the HTTPS protocol.

[1263] Step 3:

[1264] The server receives the data sent from the device, stores it in an analysis database, and uses it in subsequent analysis processes.

[1265] Input: Input data sent from the terminal.

[1266] Output: The input data stored in a database.

[1267] Specific operation: The server parses the received data in JSON format, extracts each item (photo, dimension information, equipment list, design style, etc.), and stores them in a database.

[1268] Step 4:

[1269] The server retrieves photos of the room from the database and performs image analysis using OpenCV, which uses edge detection and object recognition algorithms to identify the contours of the room and the location of furniture.

[1270] Input: A photo of a room stored in a database.

[1271] Output: Room outline and furniture position information (e.g. coordinate data).

[1272] Specific operation: Edges are detected using the OpenCV Canny algorithm, the shape of the furniture is identified using the Hough transform, and the type and location of the furniture are identified using an object recognition algorithm.

[1273] Step 5:

[1274] The server retrieves text data (dimension information, list of required equipment, information on equipment on hand, design style) from the database and performs text data analysis.

[1275] Input: Text data stored in a database.

[1276] Output: Analyzed dimensions, required equipment list, on-hand equipment information, design styles.

[1277] How it works: The server uses NLTK to parse the text data, extract dimensional information as numerical data, obtain the required equipment list and on-hand equipment information, and classify the relevant information based on the design style.

[1278] Step 6:

[1279] The server then integrates the text data and image analysis results and invokes a generative AI model to generate the optimal room layout. This process uses tools such as "DALL·E."

[1280] Input: Analyzed dimensions, required equipment list, equipment on hand, design style, image analysis results (room outline and furniture position information).

[1281] Output: Generated room layout image.

[1282] Specific operation: The server generates a prompt sentence and inputs it into the generative AI model along with the analysis data. The generative AI model then generates the optimal layout image based on this and sends it back to the server. Examples of prompt sentences include:

[1283] User input details:

[1284] Room photo

[1285] Room size: 5 meters by 4 meters

[1286] Required furniture and appliances: TV stand and carpet

[1287] Existing items: Sofa and table

[1288] Preferred style: Simple Modern

[1289] Please generate an optimal room layout image and suggest a list of furniture and appliances.

[1290] Step 7:

[1291] The server creates a list of furnishings to suggest to the user based on the generated layout image, and selects related interior and home appliances based on the analyzed design style.

[1292] Input: The generated room layout image.

[1293] Output: Proposed equipment list.

[1294] Specific behavior: The server adds related items to the list, such as TV stands and carpets that fit the simple modern style. This suggestion list is created based on the results of image and text analysis.

[1295] Step 8:

[1296] The server generates a link to an online shopping site for each proposed item, obtains the product link via API, and provides it to the user.

[1297] Input: Proposed equipment list.

[1298] Output: Purchase links for each fixture.

[1299] Specific operation: The server calls the API of the online shopping site, obtains the purchase link for each suggested item, and adds it to the list.

[1300] Step 9:

[1301] The server sends the final generated room layout image, the proposed equipment list, and a purchase link to the terminal.

[1302] Input: Generated room layout image, proposed fixture list, purchase link.

[1303] Output: The total data sent to the device.

[1304] What it does: The server converts the relevant data into a user-friendly HTML format and sends it to the device, where users can preview the interior and purchase items.

[1305] Step 10:

[1306] The terminal displays to the user an image of the final room layout, a list of suggested furnishings, and a link to purchase.

[1307] Input: Comprehensive data sent from the server.

[1308] Output: A room layout image shown to the user, a suggested furniture list, and a purchase link.

[1309] Specific operation: The user can view the layout image and suggested equipment list through the application on their device, and click on the provided link to go to the shopping site to purchase the items.

[1310] Through the above series of processing steps, the present invention provides a system that allows users to easily and efficiently realize their ideal room and purchase the necessary interior decorations and home appliances.

[1311] (Application example 1)

[1312] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1313] Traditionally, the process of selecting interior decor and home appliances to create the ideal room when moving into a new home has been extremely time-consuming and laborious. It is particularly difficult to visualize the layout of furniture and home appliances and select the appropriate items, and it is even more time-consuming to search for links to purchase each item one by one. It is also not easy to match the user's desired style with existing furniture. There is a need for a system that can solve these problems and efficiently support the creation of the ideal room.

[1314] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1315] In this invention, the server includes input means for inputting a room image, room dimension information, a list of necessary fixtures and electrical equipment, information on existing fixtures and electrical equipment, and a preferred style from the user, transmission means for transmitting data from the input means to the server, analysis means for receiving the data from the transmission means and performing image analysis and text data analysis, generation means for generating a room layout image using a generation AI based on the results of the analysis means, proposal means for generating a list of suggested fixtures and electrical equipment based on the layout generated by the generation means, link generation means for generating links to purchase pages for the fixtures and electrical equipment proposed by the proposal means, display means for transmitting and displaying the data generated by the link generation means to the user, and link acquisition means for acquiring purchase links for the suggested fixtures and electrical equipment. This allows users to easily generate their ideal room layout and purchase the necessary fixtures and electrical equipment.

[1316] A "user" is a user of the system who inputs information for selecting and arranging interior decor and home appliances for a new home.

[1317] "Room footage" refers to photos and videos of the room that users provide to the system.

[1318] "Room dimension information" is information about the physical size of a room, such as the length, width, and height of the room.

[1319] "Fixtures" refers to the interior and furniture of a room, and usually includes chairs, tables, shelves, etc.

[1320] "Electrical equipment" refers to electrical appliances and devices used in a room, such as a television or refrigerator.

[1321] "Input means" refers to a device or interface that allows a user to input room images, dimensional information, a list of fixtures and electrical equipment, etc. into the system.

[1322] "Transmission means" refers to a device or software for transmitting input data to a server.

[1323] "Analysis means" refers to functions and software for performing image analysis and text data analysis based on received data.

[1324] "Generation means" refers to functions and algorithms for generating a room layout image using generation AI based on the results of the analysis means.

[1325] The "proposal means" is a function or software that creates a list of proposed fixtures and electrical equipment based on the generated layout.

[1326] "Link generation means" refers to the functionality or software for generating links to the purchase pages of the proposed fixtures and electrical equipment.

[1327] "Display means" refers to a device or interface for displaying the generated data, suggested lists, and purchase links to the user.

[1328] "Link acquisition means" refers to the functionality or software for acquiring the purchase links for the proposed fixtures and electrical equipment.

[1329] "Style" refers to the interior design or theme that the user prefers, such as simple modern.

[1330] This invention is a system that assists users in purchasing interior decor and home appliances to create their ideal room when moving into a new home. This system uses generative AI to generate a room layout image based on user input and suggests specific fixtures and electrical equipment. The processing flow of the system's program and its specific operation are explained below.

[1331] System Overview

[1332] This system consists of a terminal and a server. The terminal sends the user's input information to the server and displays the final result to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list.

[1333] Program processing flow and operation

[1334] 1. User Input

[1335] The user uses the terminal to input a picture of the room, the room's dimensions, a list of necessary fixtures and electrical equipment, information on the fixtures and electrical equipment they have, and their preferred style.

[1336] For example, you can input a video of the living room of your new home, the dimensions of the room (5m x 4m), the need for a TV stand and carpet, the sofa and table you already have, and your preferred style (simple modern).

[1337] 2. Data Transmission

[1338] The terminal transmits the collected user input data to the server.

[1339] 3. Data Receipt and Storage

[1340] The server receives the data sent from the device and stores it in a database for analysis.

[1341] 4. Image Analysis

[1342] The server retrieves images of the room from the database and performs image analysis, using image recognition algorithms to identify the layout of the room and the location and dimensions of existing furniture. For example, edge detection and object recognition algorithms are used to identify the contours of the room and the location of furniture.

[1343] 5. Text Data Analysis

[1344] The server analyzes the room dimensions, the list of required fixtures and electrical equipment, and the information on existing fixtures and electrical equipment, thereby understanding the size of the room and the required interior specifications.

[1345] 6. Style Analysis

[1346] The server performs an analysis to identify suitable fixtures and electrical equipment based on the user's preferred style (e.g., simple modern).

[1347] 7. Data integration and input information organization

[1348] The server integrates and organizes the results of image analysis, text data analysis, and style analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[1349] 8. Layout Generation Using Generative AI

[1350] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then uses the user's input information and analysis results to automatically arrange the layout and interior.

[1351] 9. Generate a list of fixtures and electrical equipment

[1352] The server creates a list of suggested furniture and electrical equipment for the user based on the room layout image generated by the AI. For example, it adds a simple, modern TV stand or carpet to the list.

[1353] 10. Generate shopping page links

[1354] The server generates a link to a purchasing page for each proposed fixture or electrical device, possibly using an online shopping API.

[1355] 11. Data Transmission

[1356] The server sends the device an image of the final room layout, a list of proposed fixtures and electrical equipment, and a link to purchase each item.

[1357] 12. Displaying Data

[1358] The terminal displays to the user an image of the final room layout, a list of suggested fixtures and electrical equipment, and a link to purchase.

[1359] 13. User Choices and Purchases

[1360] Users can check the information displayed on their device, select the items they like from the suggested fixtures and electrical equipment, and purchase them using the provided purchase link.

[1361] Hardware and software used

[1362] Hardware:

[1363] Smartphone (with camera function)

[1364] software:

[1365] Python, Pillow, and requests

[1366] Examples:

[1367] User A takes a video of their living room with their smartphone and enters the room dimensions (5m x 4m), a list of necessary furniture (TV stand, carpet), a list of furniture they already have (sofa, table), and their preferred style (simple modern) into an input form.

[1368] The app generates the best simple modern layout image and provides purchase links for the "Simple Modern TV Stand" and "Simple Modern Carpet."

[1369] Example prompts to input to a generative AI model:

[1370] Generate the optimal room layout image based on the user's room image, dimensions, required furniture list, existing furniture information, and preferred style.

[1371] Room video: <Video from user>

[1372] Room dimensions: 5m long, 4m wide

[1373] List of necessary furniture: TV stand, carpet

[1374] Current fixtures: sofa, table

[1375] Favorite style: Simple modern

[1376] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1377] Step 1:

[1378] The user uses a terminal to input a video of the room, information about the room's dimensions, a list of necessary fixtures and electrical equipment, information about the fixtures and electrical equipment they currently have, and their preferred style. A smartphone application is used as the input method. The data the user inputs might include, for example, a video of the living room in their new home, the dimensions of the room (5m long x 4m wide), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style being simple and modern. This input information is then saved as input data.

[1379] Step 2:

[1380] The terminal sends the collected user input data to the server. This transmission method uses the HTTP protocol via the Internet. The input data is converted into JSON format and sent to the server. This data transmission procedure passes data related to the user's needs to the server.

[1381] Step 3:

[1382] The server receives the data sent from the device and stores it in a database for analysis. The server, as the receiving means, verifies the accuracy of the data and stores it in the database if there are no problems. The stored data is necessary for subsequent analysis processing and serves as the basis for realizing the user's wishes.

[1383] Step 4:

[1384] The server retrieves images of the room from the database and performs image analysis. Image analysis determines the room layout and the location and dimensions of existing furniture. Specifically, edge detection and object recognition algorithms are used to identify the contours of the room and the location of furniture. The image of the room is used as input data, and data on the layout and dimensions of furniture is output.

[1385] Step 5:

[1386] The server analyzes the room's dimensions, a list of required fixtures and electrical equipment, and information on existing fixtures and electrical equipment. It then analyzes the text data to understand the room's size and required interior specifications. This information is used as input data, and the specific specifications needed for fixture placement and selection are output.

[1387] Step 6:

[1388] The server performs an analysis to identify suitable fixtures and electrical equipment based on the user's preferred style. Analysis of the style data allows for the selection of fixtures based on the user's preferred taste, such as simple modern. Style information is used as input data, and fixtures and electrical equipment that match the user's preferences are output.

[1389] Step 7:

[1390] The server integrates and organizes the results of image analysis, text data analysis, and style analysis. This clarifies the design conditions that are optimal for the user's wishes and the characteristics of the room. Multiple analysis results are integrated and the optimal layout conditions are output.

[1391] Step 8:

[1392] The server calls the generation AI based on the integrated data and generates an optimal room layout image. The generation AI uses the user's input information and analysis results to automatically arrange the layout and interior. The following text is used as an input prompt:

[1393] Generate the optimal room layout image based on the user's room image, dimensions, required furniture list, existing furniture information, and preferred style.

[1394] Room video: <Video from user>

[1395] Room dimensions: 5m long, 4m wide

[1396] List of necessary furniture: TV stand, carpet

[1397] Current fixtures: sofa, table

[1398] Favorite style: Simple modern

[1399] Step 9:

[1400] The server creates a list of furniture and electrical equipment to suggest to the user based on the room layout image generated by the AI. A list of appropriate furniture and electrical equipment is output from the generated layout image, allowing the user to select the optimal interior.

[1401] Step 10:

[1402] The server generates a link to the purchase page for each proposed fixture or electrical device. The link generation method uses an online shopping API, which outputs a link that allows the user to directly access the purchase page.

[1403] Step 11:

[1404] The server sends the final room layout image, a list of proposed fixtures and electrical equipment, and a link to purchase each product to the terminal. The data is sent via the HTTP protocol over the Internet. The data output contains all the necessary information for the user.

[1405] Step 12:

[1406] The terminal displays the final room layout image, a list of suggested fixtures and electrical equipment, and a link to purchase. The user can review this information and evaluate the room layout. Visual information is output for optimal interior selection.

[1407] Step 13:

[1408] Users can check the information displayed on their devices, select the items they like from the suggested fixtures and electrical equipment, and purchase them using the provided purchase link. This simplifies the selection and purchase process, providing users with concrete steps to create their ideal room.

[1409] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1410] The present invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room for their new home. By incorporating an emotion engine that recognizes the user's emotions, the system can provide more appropriate suggestions based on the user's emotional state. The processing flow and specific operations of the system's program are explained below.

[1411] System Overview

[1412] This system consists of a terminal and a server. The terminal sends the user's input information to the server, analyzes the user's emotions through an emotion engine, and displays the final results to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list, adjusting them based on the user's emotional state.

[1413] Program processing flow and operation

[1414] 1. User Input

[1415] The user uses a terminal to input a photo of the room, information about the room's size, a list of the furniture and appliances they need, information about the furniture and appliances they already have, and their preferred style.The emotion engine then analyzes the user's voice, facial expression, and input speed in real time to grasp the user's emotional state.For example, a user might input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the fact that they already have a sofa and table, and their preferred style (simple modern).

[1416] 2. Data Transmission

[1417] The terminal transmits the data input by the user and the emotion data analyzed by the emotion engine to the server.

[1418] 3. Data Receipt and Storage

[1419] The server receives the data sent from the device and stores it in a database for analysis.

[1420] 4. Image Analysis

[1421] The server retrieves the room's photo data from the database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. For example, edge detection algorithms and object recognition techniques are used to detect the contours of the room and the location of furniture.

[1422] 5. Text Data Analysis

[1423] The server analyzes the text data provided by the user, including the room size, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, to understand the room size and required interior specifications.

[1424] 6. Taste and Emotion Analysis

[1425] The server performs analysis to identify suitable furniture and appliances based on the user's preferred taste (e.g., simple modern). It also adjusts the taste and style of the furniture and appliances it recommends based on the user's emotional state received from the emotion engine. For example, if the user is feeling stressed, it will suggest interior items that will help them relax.

[1426] 7. Data integration and input information organization

[1427] The server integrates and organizes the results of image analysis, text data analysis, taste analysis, and emotion analysis from the emotion engine, thereby clarifying the optimal design conditions for the user's wishes and the characteristics of the room.

[1428] 8. Layout Generation Using Generative AI

[1429] The server then calls a generative AI based on the integrated data to generate an optimal room layout image. This generative AI automatically arranges furniture and appliances to create the optimal layout based on the user's input data, analysis results, and emotional state.

[1430] 9. Generate a list of interior, furniture, and home appliances

[1431] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a TV stand or carpet with a simple, modern design to the list. The server also takes into account the user's emotional state to further optimize the options.

[1432] 10. Generate shopping page links

[1433] The server generates a link to the purchase page for each suggested piece of furniture or home appliance, taking into account the size and placement information of each product. Based on the analysis results of the emotion engine, the server makes suggestions that will increase user satisfaction.

[1434] 11. Data Transmission

[1435] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[1436] 12. Displaying Data

[1437] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[1438] 13. User Choices and Purchases

[1439] Users can select the furniture or appliances they like from the suggested list displayed on their device and complete the purchase using the provided link. At this time, they can also check the specific interior layout while making their selection.

[1440] This concludes the description of the embodiment of the invention. This system not only allows users to efficiently and effectively select and arrange furniture and home appliances that suit their room and preferences, but also provides links to purchase each item. Furthermore, by adjusting the suggestions based on the user's emotional state, it is possible to create a room that is more satisfying.

[1441] The processing flow will be explained below.

[1442] Step 1:

[1443] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they currently have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style (simple modern).

[1444] Step 2:

[1445] The device recognizes the user's emotional state through the user's tone of voice, facial expression, and input speed. The emotion engine analyzes this information and estimates the user's emotional state, for example, determining whether the user is relaxed or nervous.

[1446] Step 3:

[1447] The terminal transmits all data input by the user and emotion data analyzed by the emotion engine to the server.

[1448] Step 4:

[1449] The server receives the data sent from the device and stores it in a database for analysis.

[1450] Step 5:

[1451] The server retrieves the room's photo data from the database and performs image analysis, using edge detection algorithms and object recognition techniques to identify the room layout and the location and dimensions of existing furniture. For example, it recognizes the boundaries between the floor and walls of the room and identifies the placement of existing furniture.

[1452] Step 6:

[1453] The server analyzes the room size information, the list of necessary furniture and appliances, and the information on the furniture and appliances currently available as text data. This clarifies the specific dimensions of the room and the interior requirements. For example, it determines the size of the TV stand and carpet that are appropriate for the living room.

[1454] Step 7:

[1455] The server analyzes the user's preferred taste (e.g., simple modern) and identifies suitable furniture and home appliances. It also adjusts the taste and style of the furniture and home appliances it recommends based on the emotional data obtained from the emotion engine. For example, if the user wants to relax, it will suggest furniture in soft colors.

[1456] Step 8:

[1457] The server integrates the results of image analysis, text data analysis, taste analysis, and emotion analysis from the emotion engine to determine optimal design conditions.

[1458] Step 9:

[1459] The server uses the integrated data to call a generative AI to generate an optimal room layout image. This generative AI automatically arranges furniture and appliances based on the user's input data and analysis results, and creates the optimal layout by taking emotional data into consideration.

[1460] Step 10:

[1461] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a TV stand or carpet with a simple, modern design to the list, reflecting the results of sentiment analysis.

[1462] Step 11:

[1463] The server generates links to purchase pages for the suggested furniture and appliances, taking into account the size and placement of each product and adjusting them according to the user's emotional state.

[1464] Step 12:

[1465] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[1466] Step 13:

[1467] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[1468] Step 14:

[1469] Users can select their favorite furniture or appliances from the suggested list displayed on their device and purchase them using the provided link, or they can rearrange the proposed interior. Here, they can make their final selection while checking the specific interior layout.

[1470] Example 2

[1471] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1472] Conventional purchasing support systems for room interiors and home appliances require users to manually sort through large amounts of information and determine the optimal interior layout, which is extremely time-consuming. Furthermore, since suggestions are not based on the user's emotional state, this can result in low user satisfaction. There is a need for a system that overcomes these drawbacks, reduces the user's time and effort, and provides suggestions that are in line with individual emotions.

[1473] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1474] In this invention, the server includes: input means for inputting image data of the room, room dimension information, a list of necessary items, information on items on hand, and a preferred interior style from the user; communication means for transmitting the data from the input means to a central processing unit; analysis means for receiving the data from the communication means and performing image analysis and text data analysis; emotion recognition means for grasping the user's emotional state; generation means for generating a room layout image using a generative AI model based on the results of the analysis means and the emotion recognition means; suggestion means for generating a list of suggested items based on the layout generated by the generation means; link generation means for generating links to purchase pages for the items suggested by the suggestion means; and display means for transmitting the data generated by the link generation means to the user and displaying it. This allows the user to receive suggestions for optimal interior layout and items based on their emotional state.

[1475] "User" refers to a person who uses this system to receive suggestions for room interiors and home appliances.

[1476] "Input means" refers to a device or software that allows a user to input image data of a room, dimensional information, a list of necessary items, information on items on hand, and a preferred interior style.

[1477] "Communication means" refers to a method or device for transmitting data from an input means to a central processing unit.

[1478] "Analysis means" refers to devices and algorithms that perform image analysis and text data analysis based on received data.

[1479] "Emotion recognition means" refers to devices or software that analyze a user's voice, facial expressions, input speed, etc. to understand their emotional state.

[1480] "Generation means" refers to a device or software for generating a room layout image using a generative AI model based on the results of the analysis means and emotion recognition means.

[1481] The "suggestion means" refers to a method or apparatus for creating a suggested item list based on the layout generated by the generation means.

[1482] The "link generating means" refers to a method or device for generating a link to a purchase page for the item suggested by the suggesting means.

[1483] The "display means" refers to a device or software for transmitting the data generated by the link generation means to the user and visually presenting it to the user.

[1484] A "generative AI model" refers to an artificial intelligence algorithm that automatically creates the optimal room layout based on user input data and analysis results.

[1485] The present invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room when moving into a new home. This system is configured using a terminal and a server. The detailed configuration and operation of the system are described below.

[1486] overview

[1487] This system assists users in the process of selecting the best interior and home appliances for their new home. The system incorporates an emotion engine to constantly grasp the user's emotional state and make more appropriate suggestions. Users can input data using their device to review and purchase the suggested interior and home appliances.

[1488] Specific actions

[1489] 1. User Input

[1490] The user uses the device to input image data of the room they will be living in, the room's dimensions, a list of items they need, information about items they already have, and their preferred interior style. The emotion engine analyzes the user's voice, facial expressions, and input speed in real time to grasp their emotional state.

[1491] Example: Take a photo of the living room of your new home with your smartphone and upload it. Then, enter the room dimensions (5m x 4m), items you need (TV stand, carpet), furniture you already have (sofa, table), and your preferred interior style (simple modern) in the input fields.

[1492] 2. Data Transmission

[1493] The terminal transmits data input by the user and emotion data analyzed by the emotion engine to the server. This data includes images, text, and emotion analysis results.

[1494] 3. Data Receipt and Storage

[1495] The server receives the data sent from the device and stores it in a database for analysis. The stored data includes photos of the room, text data, and emotion analysis data.

[1496] 4. Image Analysis

[1497] The server retrieves photos of the room from a database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. Techniques used include edge detection and object recognition algorithms.

[1498] Example: The server uses the image analysis library OpenCV to extract the contours of a room and the position of furniture using an edge detection algorithm.

[1499] 5. Text Data Analysis

[1500] The server performs text analysis of the room dimension information, list of required items, and information on items on hand provided by the user to understand the size of the room and the specifications of the required items.

[1501] 6. Taste and Emotion Analysis

[1502] The server performs analysis to identify suitable items based on the user's preferred interior style (e.g., simple modern), and adjusts the style of the suggested items based on the user's emotional state received from the emotion engine.

[1503] Example: If the user is feeling stressed, suggest interior design that will help them relax.

[1504] 7. Data integration and input information organization

[1505] The server integrates and organizes the image analysis results, text data analysis results, taste analysis results, and emotion analysis results from the emotion engine.

[1506] 8. Layout Generation Using Generative AI Models

[1507] The server calls up a generative AI model based on the integrated data and generates an optimal room layout image.

[1508] Example prompt for a generative AI model:

[1509] "Generate a room layout in a simple, modern interior style using the user's input data: room dimensions (5m x 4m), items on hand (sofa and table), and a list of items needed (TV stand and carpet). The user's emotional state should be relaxed."

[1510] 9. Generate a list of interior items

[1511] The server creates a list of items to suggest to the user based on the room layout image generated by the generative AI model.

[1512] 10. Generate shopping page links

[1513] The server generates a link to the purchase page for each suggested item, taking into account the analysis results of the emotion engine to ensure a high level of user satisfaction.

[1514] 11. Data Transmission

[1515] The server sends the generated final data (room layout image, suggested item list, and purchase link) to the terminal.

[1516] 12. Displaying Data

[1517] The terminal displays the final room layout image, a list of suggested items, and a link to purchase them to the user, allowing the user to select the interior design while reviewing them.

[1518] 13. User Choices and Purchases

[1519] Users can select the items they like from the suggested list displayed on their device and complete the purchase using the provided link, while also checking the specific interior layout.

[1520] This system not only allows users to efficiently and effectively select and arrange interior and home appliances that suit their room and preferences, but also provides links to purchase each item. Furthermore, by adjusting the suggestions based on the user's emotional state, it is possible to create a room that is more satisfying.

[1521] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1522] Step 1:

[1523] User Input

[1524] The user uses the device to input image data of the room, room dimensions, a list of items needed, information about items they have on hand, and their preferred interior style. The emotion engine analyzes voice, facial expressions, and input speed in real time to grasp the user's emotional state.

[1525] Input: room image data, room dimensions, list of necessary items, information on items on hand, preferred interior style

[1526] Output: User input data and emotion data stored on the device

[1527] How it works: The user takes a photo of the living room of their new home with their smartphone and uploads it to the application. Then, they fill in the input fields with the room dimensions (5m x 4m), items they need (TV stand, carpet), furniture they already have (sofa, table), and their preferred interior style (simple modern).

[1528] Step 2:

[1529] Data transmission

[1530] The terminal transmits data input by the user and emotion data analyzed by the emotion engine to the server.

[1531] Input: User input data and emotion data stored on the device

[1532] Output: User input data and emotion data sent to the server

[1533] Specific operation: The terminal uses the API to send input data in JSON format to the server.

[1534] Step 3:

[1535] Data reception and storage

[1536] The server receives the data sent from the device and stores it in a database for analysis.

[1537] Input: User input data and emotion data sent to the server

[1538] Output: User data and emotion data stored in a database

[1539] Specific operation: The server calls the database save function to save the received data. For example, the image data of the room is saved in Blob format, and the text data is saved in a structured database.

[1540] Step 4:

[1541] Image analysis

[1542] The server retrieves image data of the room from the database and performs image analysis to determine the room layout, the location of existing furniture, and its dimensions.

[1543] Input: Image data of the room stored in the database

[1544] Output: Room layout, existing furniture locations, dimensions

[1545] Specific operation: The server uses the OpenCV library to extract the room contours and furniture positions using an edge detection algorithm.

[1546] Step 5:

[1547] Text Data Analysis

[1548] The server performs text analysis of the room dimension information, list of required items, and information on items on hand provided by the user to understand the size of the room and the specifications of the required items.

[1549] Input: Room dimensions stored in the database, list of required items, and information on items on hand

[1550] Output: Analyzed room size, required item specifications

[1551] What it does: The server uses natural language processing (NLP) tools to parse the text input data and get the room dimensions and a list of required items.

[1552] Step 6:

[1553] Taste and emotion analysis

[1554] The server performs analysis to identify suitable items based on the user's preferred interior style (e.g., simple modern), and adjusts the style of the suggested items based on the user's emotional state received from the emotion engine.

[1555] Input: User's preferred interior style, sentiment analysis results

[1556] Output: Adjusted product list and style suggestions

[1557] How it works: The server uses a taste analysis algorithm to search for furniture that suits a simple modern style, and customizes suggestions based on the results of sentiment analysis.

[1558] Step 7:

[1559] Data integration and input organization

[1560] The server integrates and organizes the results of image analysis, text data analysis, taste analysis, and emotion analysis.

[1561] Input: Image analysis results, text data analysis results, taste analysis results, emotion analysis results

[1562] Output: Integrated interior proposal data

[1563] Specific operation: The server integrates these analysis results into a single data structure, compiling all the information necessary to propose the optimal interior design.

[1564] Step 8:

[1565] Layout generation using generative AI models

[1566] The server calls up a generative AI model based on the integrated data and generates an optimal room layout image.

[1567] Input: Integrated interior proposal data

[1568] Output: Generated room layout image

[1569] Specific operation: The server sends a prompt to the generative AI model (e.g., "Generate a simple, modern living room layout based on the user's input data") and retrieves the returned results.

[1570] Step 9:

[1571] Generate a list of interior items

[1572] The server creates a list of items to suggest to the user based on the room layout image generated by the generative AI model.

[1573] Input: Generated room layout image

[1574] Output: List of suggested items

[1575] How it works: The server checks a database of furniture and appliances to create a list of items that match the layout image.

[1576] Step 10:

[1577] Generate shopping page links

[1578] The server generates a link to a purchase page for each suggested item, taking into account the analysis results of the emotion engine to maximize user satisfaction.

[1579] Input: List of suggested items, sentiment analysis results

[1580] Output: Link to purchase page

[1581] What it does: The server collects online shop links for each item and adds the links to a list for easy user access.

[1582] Step 11:

[1583] Sending data

[1584] The server sends the generated final data (room layout image, suggested item list, and purchase link) to the terminal.

[1585] Input: Final data (room layout image, suggested item list, purchase link)

[1586] Output: Final data sent to the terminal

[1587] Specific operation: The server uses the API to send the final data to the terminal.

[1588] Step 12:

[1589] Viewing Data

[1590] The terminal displays the final room layout image, a list of suggested items, and a link to purchase them to the user, allowing the user to select the interior while reviewing them.

[1591] Input: Final data sent from the server

[1592] Output: Layout image displayed to the user, item list, purchase link

[1593] Specific operation: The device displays the acquired data on the application's UI, making it easy for the user to view.

[1594] Step 13:

[1595] User Choice and Purchase

[1596] The user selects the item they like from a list of suggestions displayed on their device and completes the purchase using the provided link.

[1597] Input: Displayed suggestion list and purchase link

[1598] Output: Purchased item

[1599] Specific operation: The user clicks on the purchase link, goes to the online shop page and purchases the product.

[1600] This allows users to efficiently select and arrange interior and home appliances that suit their room and preferences, and receive suggestions based on their emotional state.

[1601] (Application example 2)

[1602] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1603] Conventional interior design suggestion systems have difficulty in making suggestions that take into account the user's emotional state, which results in a failure to increase user satisfaction. Furthermore, when purchasing in a physical store, they are unable to make optimal suggestions based on real-time interior layout or emotions, which means it takes a lot of time and effort for users to realize their ideal room.

[1604] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1605] In this invention, the server includes: emotion analysis means for analyzing the user's emotional state and adjusting the proposal content based on that state; input means for inputting from the user an image of the room, room dimension information, a list of necessary household goods and home appliances, information on household goods and home appliances currently in use, and a preferred design; transmission means for transmitting data from the input means to a central processing unit; analysis means for receiving the data from the transmission means and performing image analysis and text data analysis; generation means for generating a room layout image using a generation AI based on the results of the analysis means; proposal means for generating a list of proposed household goods and home appliances based on the layout generated by the generation means; link generation means for generating links to purchasing pages for the household goods and home appliances proposed by the proposal means; and display means for transmitting and displaying the data generated by the link generation means to the user. This enables optimal interior proposals in real time while taking the user's emotional state into consideration, and facilitates smooth purchases in physical stores.

[1606] "User" refers to a person who uses the system.

[1607] "Room image" refers to a photo of a room taken by a user.

[1608] "Room dimension information" refers to information about the length, width, and height of a room.

[1609] "List of necessary household goods and home appliances" refers to a list of household goods and home appliances that the user wants to purchase.

[1610] "Information about existing household goods and home appliances" refers to information about household goods and home appliances that the user already owns.

[1611] "Preferred design" refers to the interior style or theme that the user prefers.

[1612] "Input means" refers to a device or method by which a user inputs information into a system.

[1613] The "transmission means" refers to a function for transmitting data acquired from the input means to the central processing unit.

[1614] "Analysis means" refers to a device or method that analyzes received data and performs image and text data analysis.

[1615] "Generation means" refers to the function of generating a room layout image using generation AI based on the results of the analysis means.

[1616] The "suggestion means" refers to a function for generating a list of suggested household goods and home appliances based on the arrangement generated by the generation means.

[1617] The "link generating means" refers to a function for generating a link to a purchase page for the household goods / home appliances suggested by the suggesting means.

[1618] The "display means" refers to a function that displays the data generated by the link generation means to the user.

[1619] "Emotion analysis means" refers to a function that analyzes the user's emotional state and adjusts the content of suggestions based on that state.

[1620] "Central Processing Unit" refers to the main unit for processing, analyzing and generating user input data.

[1621] The present invention is a system that supports the purchase of interior decorations and home appliances to realize the ideal room when a user moves into a new home. By combining this system with emotion analysis means, it is possible to make more appropriate suggestions according to the user's emotional state.

[1622] System configuration

[1623] This system is mainly composed of terminals and servers, and is explained in detail below.

[1624] Terminal

[1625] The terminal is a device that inputs information through user operation and sends it to the server. The terminal is equipped with hardware such as a camera, microphone, and display, and performs the following functions:

[1626] Input means: The user uses the terminal to input an image of the room, room dimensions, a list of necessary household goods and appliances, information about the household goods and appliances they already have, and their preferred design.

[1627] Emotion analysis means: When a user inputs information, voice, facial expressions, input speed, etc. are analyzed in real time to understand the user's emotional state.

[1628] Transmission method: The input data and the emotion analysis results are sent to the server.

[1629] server

[1630] The server receives data sent from the terminal, analyzes it, generates it, and makes suggestions. It has the following functions:

[1631] Analysis methods: Image analysis and text data analysis are performed. Image analysis uses edge detection algorithms and object recognition techniques to identify the room layout and the location of existing furniture. Text data analysis analyzes the room dimensions and specifications of required furniture.

[1632] Generation method: Based on the analysis results, a generative AI is used to generate an optimal room layout image.

[1633] Proposal method: Based on the generated layout image, a list of suggested household goods and home appliances is generated.

[1634] Link generation method: Generate a link to the purchase page for the suggested household goods and appliances.

[1635] Display means: The generated data is sent to the terminal and displayed to the user.

[1636] Detailed System Operation

[1637] Program processing

[1638] The server uses image processing libraries such as OpenCV to analyze the room photos sent by the user. It also uses a natural language processing engine to analyze text data. It uses a generative AI model (such as GAN or VAE) to generate an optimal room layout image based on the analysis results.

[1639] Emotion analysis means

[1640] The emotion analysis method uses an emotion recognition engine (such as Microsoft Azure's Emotion API or Google Cloud's Vision API) to analyze the user's emotional state in real time from their voice and facial expressions. This allows the system to suggest interior designs that will help the user relax if they are feeling stressed, or that will help them maintain their excitement if they are enjoying themselves.

[1641] Specific examples

[1642] For example, suppose a user takes a photo of the living room of their new home and inputs the room dimensions (5m x 4m), the furniture they need (TV stand and carpet), the furniture they already have (sofa and table), and their preferred design (simple modern). Based on this information and the results of sentiment analysis, the server uses a generative AI model to generate an optimal layout image. It then creates a list of suggested furniture and appliances and generates a link to the purchasing page to display it to the user.

[1643] Prompt Sentence Examples

[1644] "Generate an optimal room layout image based on a photo of the room, size information, a list of necessary furniture and appliances, information on the furniture and appliances currently in use, and the user's preferred design (e.g., simple modern). Please also include suggestions based on the user's emotional state (e.g., relaxation)."

[1645] This system not only allows users to efficiently and effectively select and arrange furniture and appliances that suit their room and preferences, but also provides links to purchase each item. Furthermore, by adjusting the suggestions based on the user's emotional state, it is possible to create a room that is more satisfying.

[1646] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1647] Step 1:

[1648] The user uses the terminal to input a picture of the room, dimension information, a list of necessary household goods and appliances, information about the household goods and appliances they already have, and their preferred design.

[1649] Input: Room photo, dimensions, list of necessary furniture and appliances, information on existing furniture and appliances, preferred design

[1650] Output: Input data (a series of information)

[1651] Specific operation: The user uses the application on their smartphone or tablet to take a photo of the room using the camera function and fill out an input form with various information. This information is temporarily stored on the device as integrated data.

[1652] Step 2:

[1653] The terminal uses an emotion analysis means to analyze the user's emotional state in real time from their voice and facial expressions.

[1654] Input: User's voice, facial expressions

[1655] Output: Sentiment analysis data

[1656] How it works: The device uses a microphone and camera to capture the user's voice and facial expressions. This data is then fed into an emotion analysis engine (e.g., Emotion API) to analyze the user's emotional state (e.g., stress, joy) in real time.

[1657] Step 3:

[1658] The terminal transmits the input data and the emotion analysis results to the server.

[1659] Input: Input data, sentiment analysis data

[1660] Output: Data sent to the server

[1661] Specific operation: The device combines the user's input data and emotion analysis data and sends it to the server as a single data packet. The communication protocol is HTTPS or similar, ensuring secure transmission.

[1662] Step 4:

[1663] The server stores the received data in a database for analysis and performs image analysis and text data analysis.

[1664] Input: Data sent to the server

[1665] Output: Image analysis results, text data analysis results

[1666] Specific operation: The server stores the received data in an analysis database. It then analyzes the room photo using image processing libraries such as OpenCV, and performs edge detection algorithms and object recognition. At the same time, it uses a text data analysis engine to analyze the dimensions and furniture list.

[1667] Step 5:

[1668] Based on the analysis results, the server uses a generative AI model to generate an optimal room layout image.

[1669] Input: Image analysis results, text data analysis results, emotion analysis data

[1670] Output: Generated aligned image

[1671] How it works: The server takes the analysis results and emotion data as input and provides them to a generative AI model (e.g., GAN), which then generates an optimal room layout image that matches the user's preferences.

[1672] Step 6:

[1673] The server generates a list of suggested household goods and home appliances based on the generated layout image.

[1674] Input: Generated aligned image

[1675] Output: List of suggested household goods and appliances

[1676] How it works: The server analyzes the layout images output by the generative AI model and automatically generates a list of optimal proposals, including detailed information such as the size and design of furniture and appliances.

[1677] Step 7:

[1678] The server generates a link to the purchase page for the suggested household goods and home appliances.

[1679] Input: List of suggested household goods and appliances

[1680] Output: Link to purchase page

[1681] How it works: For each item in the suggestion list, the server searches an online shopping database and generates a link to the page where it can be purchased, optimized to take into account the user's emotional state.

[1682] Step 8:

[1683] The server transmits the generated data to the terminal, which displays it to the user.

[1684] Input: Purchase link, proposal list, placement image

[1685] Output: What is displayed to the user

[1686] Specific operation: The server sends a link to the purchase page, a list of suggestions, and layout images to the device. The device receives this data and displays it in a format that is easy for the user to view. The user can then select and purchase interior items based on the displayed information.

[1687] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1688] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1689] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1690] [Fourth embodiment]

[1691] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1692] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1693] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1694] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1695] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1696] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1697] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1698] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1699] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1700] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1701] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1702] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1703] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1704] This invention is a system that assists users in purchasing interior decor and home appliances to create the ideal room when moving into a new home. This system uses a generative AI to generate a room layout image based on user input and suggests specific interior decor and home appliances. The processing flow of the system's program and its specific operation are explained below.

[1705] System Overview

[1706] This system consists of a terminal and a server. The terminal sends the user's input information to the server and displays the final result to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list.

[1707] Program processing flow and operation

[1708] 1. User Input

[1709] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they currently have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style (simple modern).

[1710] 2. Data Transmission

[1711] The terminal transmits the collected user input data to the server.

[1712] 3. Data Receipt and Storage

[1713] The server receives the data sent from the device and stores it in a database for analysis.

[1714] 4. Image Analysis

[1715] The server retrieves a photo of the room from the database and performs image analysis, using image recognition algorithms to identify the room's layout and the location and dimensions of existing furniture, for example, using edge detection and object recognition algorithms to identify the contours of the room and the location of furniture.

[1716] 5. Text Data Analysis

[1717] The server analyzes the room size information, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, thereby understanding the size of the room and the required interior specifications.

[1718] 6. Taste Analysis

[1719] The server performs an analysis to identify suitable furniture and appliances based on the user's preferred style (e.g., simple modern).

[1720] 7. Data integration and input information organization

[1721] The server integrates and organizes the results of image analysis, text data analysis, and taste analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[1722] 8. Layout Generation Using Generative AI

[1723] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then uses the user's input information and analysis results to automatically arrange the layout and interior.

[1724] 9. Generate a list of interior, furniture, and home appliances

[1725] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the generative AI. For example, it adds a simple, modern TV stand or carpet to the list.

[1726] 10. Generate shopping page links

[1727] The server generates a link to a purchasing page for each suggested piece of furniture or appliance, possibly using an online shopping API.

[1728] 11. Data Transmission

[1729] The server sends the device an image of the final room layout, a list of suggested furniture and appliances, and a link to purchase each item.

[1730] 12. Displaying Data

[1731] The device displays to the user an image of the final room layout, a list of suggested furniture and appliances, and a link to purchase.

[1732] 13. User Choices and Purchases

[1733] Users can check the information displayed on their device, select the furniture or home appliances they like from the suggested items, and purchase them using the provided purchase link.

[1734] This concludes the description of the embodiment of the invention. This system not only allows users to efficiently and effectively select and arrange furniture and home appliances that suit their room and preferences, but also provides links to purchase each item at the same time. This allows users to easily create their ideal room.

[1735] The processing flow will be explained below.

[1736] Step 1:

[1737] The user uses the terminal to input a photo of the room, information about the room's size, a list of necessary furniture and appliances, information about the furniture and appliances they already have, and their preferred style. For example, they can input a photo of the living room of their new home, the room size (5m x 4m), that they need a TV stand and carpet, that they already have a sofa and table, and that their preferred style is simple and modern.

[1738] Step 2:

[1739] The terminal collectively transmits the data input by the user to the server.

[1740] Step 3:

[1741] The server receives the data sent from the device and stores it in a database for analysis.

[1742] Step 4:

[1743] The server retrieves the room's photo data from the database and performs image analysis to determine the room's layout, the location of existing furniture, and its dimensions. For example, edge detection algorithms and object recognition techniques are used to detect the contours of the room and the location of furniture.

[1744] Step 5:

[1745] The server analyzes the text data provided by the user, including the room size, the list of necessary furniture and appliances, and the information on the furniture and appliances currently in use, to understand the room size and required interior specifications.

[1746] Step 6:

[1747] The server performs analysis to identify suitable furniture and appliances based on the user's preferred style (e.g., simple modern).

[1748] Step 7:

[1749] The server integrates and organizes the results of image analysis, text data analysis, and taste analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[1750] Step 8:

[1751] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then automatically arranges furniture and appliances based on the user's input data and analysis results to create the optimal layout.

[1752] Step 9:

[1753] The server creates a list of furniture and appliances to suggest to the user based on the room layout image generated by the AI. For example, it adds a TV stand or carpet with a simple, modern design to the list.

[1754] Step 10:

[1755] The server generates a link to the online shopping site for each proposed piece of furniture or appliance, taking into account the size and placement information of each item.

[1756] Step 11:

[1757] The server sends the generated final data (room layout image, suggested furniture and appliance list, and purchase link) to the terminal.

[1758] Step 12:

[1759] The device displays the final room layout image, a list of suggested furniture and appliances, and a link to purchase the item, allowing the user to review and select their interior design.

[1760] Step 13:

[1761] Users can select the furniture or appliances they like from the suggested list displayed on their device and complete the purchase using the provided link. At this time, they can also check the specific interior layout while making their selection.

[1762] Example 1

[1763] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1764] In conventional interior and home appliance purchasing support systems, it has been difficult for users to effectively plan the layout, interior, and placement of home appliances in a room. In particular, there was a lack of a way to generate an optimal layout while taking into account the user's preferences, existing furniture, and room size. In addition, generating purchase links had to be done manually, which was a cumbersome and time-consuming task for users. These challenges made it difficult to efficiently create the ideal room.

[1765] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1766] In this invention, the server includes input means for allowing a user to input a photo of the room, room dimension information, a list of necessary fixtures, information about fixtures on hand, and a preferred design style, transmission means for transmitting data from the input means to the server, analysis means for receiving the data from the transmission means and performing image analysis and text data analysis, generation means for generating a room layout image using a generative AI model based on the results of the analysis means, proposal means for generating a proposed fixture list based on the layout generated by the generation means, link generation means for generating links to purchasing pages for the fixtures proposed by the proposal means, and display means for transmitting and displaying the data generated by the link generation means to the user. This enables users to efficiently design a room with an ideal layout based on their preferences and easily purchase the necessary fixtures.

[1767] A "user" is an individual or organization that uses the system to receive support for room layout, interior design, and purchasing home appliances.

[1768] A "terminal" is a device used by a user to provide input information to the system, and includes a PC, smartphone, tablet, etc.

[1769] The "server" is a central processing unit that receives data sent from users, analyzes it, and generates layout images and proposal lists.

[1770] The "input means" is an interface that allows the user to input a photo of the room, dimension information, a list of necessary equipment, information on equipment that is on hand, and a preferred design style.

[1771] The "transmission means" is a means for transmitting the user's input data from the terminal to the server.

[1772] The "analysis means" is a means for analyzing received data in the server and performing image analysis and text data analysis.

[1773] The "generation means" is a means for generating a room layout image using a generative AI model based on the results of the analysis means.

[1774] The "proposing means" is a means for generating an equipment list to be proposed to the user based on the layout generated by the generating means.

[1775] The "link generating means" is a means for generating a link to a purchasing page for the equipment suggested by the suggesting means.

[1776] The "display means" is a means for transmitting the data generated by the link generation means to the user and displaying it.

[1777] "Image analysis" is a technology that processes photos of a room sent by a user and identifies the outline of the room and the location of furniture.

[1778] "Text data analysis" is a technology that analyzes the room dimension information, required equipment list, equipment information on hand, and preferred design style sent by the user.

[1779] A "generative AI model" is an artificial intelligence model that generates optimal room layout images based on user input information and analysis results.

[1780] A "prompt sentence" is an input sentence used to give instructions to a generative AI model.

[1781] "Fixtures" is a general term for decorative and functional items, including interior furnishings and home appliances.

[1782] "Design style" is a concept that refers to a particular theme or aesthetic of room decoration or interior that a user prefers.

[1783] This invention is a system that efficiently supports the purchase of interior decor and home appliances to create an ideal room when a user moves into a new residence. This system is composed of a terminal and a server. The terminal sends the user's input information to the server, and the server uses a generative AI model to generate a room layout image and a proposal list. The final data is then provided to the user.

[1784] Input Method

[1785] Users use their device to input a photo of their room, its dimensions, a list of necessary furnishings, information about the furnishings they already have, and their preferred design style. For example, a user can take a photo of their living room with their smartphone and upload it to the application. They can also input the room dimensions (5 meters long and 4 meters wide), select a TV stand and carpet as necessary items, input the sofa and table they already have, and set their preferred design style to simple modern.

[1786] Transmission method

[1787] The terminal sends the user's input data collected through the input means to the server. This transmission is securely performed using the HTTPS protocol. The input data is converted to JSON format, encrypted, and sent to the server.

[1788] Analysis means

[1789] The server receives data sent from the device and stores it in a database for analysis. For image analysis, the open-source image processing library "OpenCV" is used, using edge detection and object recognition algorithms to identify the outline of the room and the location of furniture. For text data analysis, the "NLTK" library is used to analyze room dimensions and a list of necessary equipment. For taste analysis, generative AI models such as "GPT-3" are used.

[1790] generation means

[1791] Based on the analysis results, the server uses a generative AI model to generate a room layout image. Specifically, it uses a generative AI model such as "DALL·E" and generates the optimal room layout image by inputting a text prompt. The following text is an example of a prompt that can be used:

[1792] Example prompt sentence:

[1793] User input details:

[1794] Room photo

[1795] Room size: 5 meters by 4 meters

[1796] Required furniture and appliances: TV stand and carpet

[1797] Existing items: Sofa and table

[1798] Preferred style: Simple Modern

[1799] Please generate an optimal room layout image and suggest a list of furniture and appliances.

[1800] Proposal means

[1801] The server then creates a list of suggested furnishings for the user based on the layout image. For example, it adds a TV stand and carpet suitable for a simple modern style to the list. This suggestion is based on the analyzed user preferences.

[1802] Link Generation Methods

[1803] For each proposed item, the server generates a link to an online shopping site to purchase the item. Specifically, it retrieves the product link using the API of a popular online marketplace.

[1804] Display means

[1805] The server sends the final room layout image, the proposed equipment list, and a purchase link for each item to the terminal, which then displays this information to the user. For example, the user can view the layout image and the proposed equipment list through the application interface and click the purchase link to go to a shopping site.

[1806] User Choice and Purchase

[1807] Users can select their favorite items based on the suggested information and complete the purchase process using the provided purchase link, allowing them to efficiently create their ideal room.

[1808] The system of the present invention allows users to simultaneously obtain layout suggestions that suit their room and preferences, a list of necessary fixtures, and links to purchase each item, making it easy for users to create their ideal room.

[1809] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1810] Step 1:

[1811] The user uses the terminal to input a photo of the room, room dimensions, a list of necessary equipment, information on equipment they have on hand, and their preferred design style. The information entered by the user is entered into an input form using text boxes and selection lists. The entered data is stored in the system's internal memory.

[1812] Input: The user inputs a photo of the room, dimension information, a list of necessary equipment, information on equipment on hand, and design style through the terminal.

[1813] Output: Input data stored in internal memory.

[1814] Specific operation: The user takes a photo of their living room with their smartphone camera and uploads it to the application. They also input the room dimensions (5 meters long x 4 meters wide), select a TV stand and carpet as necessary items, input information about their existing sofa and table, and set their preferred design style to simple modern.

[1815] Step 2:

[1816] The terminal sends the data entered by the user to the server, where the transmission is encrypted via the HTTPS protocol, ensuring reliable communication.

[1817] Input: Input data stored in the internal memory.

[1818] Output: The input data sent to the server.

[1819] Specific behavior: When the user taps the "Submit" button, the entered data is converted to JSON format and sent to the server using the HTTPS protocol.

[1820] Step 3:

[1821] The server receives the data sent from the device, stores it in an analysis database, and uses it in subsequent analysis processes.

[1822] Input: Input data sent from the terminal.

[1823] Output: The input data stored in a database.

[1824] Specific operation: The server parses the received data in JSON format, extracts each item (photo, dimension information, equipment list, design style, etc.), and stores them in a database.

[1825] Step 4:

[1826] The server retrieves photos of the room from the database and performs image analysis using OpenCV, which uses edge detection and object recognition algorithms to identify the contours of the room and the location of furniture.

[1827] Input: A photo of a room stored in a database.

[1828] Output: Room outline and furniture position information (e.g. coordinate data).

[1829] Specific operation: Edges are detected using the OpenCV Canny algorithm, the shape of the furniture is identified using the Hough transform, and the type and location of the furniture are identified using an object recognition algorithm.

[1830] Step 5:

[1831] The server retrieves text data (dimension information, list of required equipment, information on equipment on hand, design style) from the database and performs text data analysis.

[1832] Input: Text data stored in a database.

[1833] Output: Analyzed dimensions, required equipment list, on-hand equipment information, design styles.

[1834] How it works: The server uses NLTK to parse the text data, extract dimensional information as numerical data, obtain the required equipment list and on-hand equipment information, and classify the relevant information based on the design style.

[1835] Step 6:

[1836] The server then integrates the text data and image analysis results and invokes a generative AI model to generate the optimal room layout. This process uses tools such as "DALL·E."

[1837] Input: Analyzed dimensions, required equipment list, equipment on hand, design style, image analysis results (room outline and furniture position information).

[1838] Output: Generated room layout image.

[1839] Specific operation: The server generates a prompt sentence and inputs it into the generative AI model along with the analysis data. The generative AI model then generates the optimal layout image based on this and sends it back to the server. Examples of prompt sentences include:

[1840] User input details:

[1841] Room photo

[1842] Room size: 5 meters by 4 meters

[1843] Required furniture and appliances: TV stand and carpet

[1844] Existing items: Sofa and table

[1845] Preferred style: Simple Modern

[1846] Please generate an optimal room layout image and suggest a list of furniture and appliances.

[1847] Step 7:

[1848] The server creates a list of furnishings to suggest to the user based on the generated layout image, and selects related interior and home appliances based on the analyzed design style.

[1849] Input: The generated room layout image.

[1850] Output: Proposed equipment list.

[1851] Specific behavior: The server adds related items to the list, such as TV stands and carpets that fit the simple modern style. This suggestion list is created based on the results of image and text analysis.

[1852] Step 8:

[1853] The server generates a link to an online shopping site for each proposed item, obtains the product link via API, and provides it to the user.

[1854] Input: Proposed equipment list.

[1855] Output: Purchase links for each fixture.

[1856] Specific operation: The server calls the API of the online shopping site, obtains the purchase link for each suggested item, and adds it to the list.

[1857] Step 9:

[1858] The server sends the final generated room layout image, the proposed equipment list, and a purchase link to the terminal.

[1859] Input: Generated room layout image, proposed fixture list, purchase link.

[1860] Output: The total data sent to the device.

[1861] What it does: The server converts the relevant data into a user-friendly HTML format and sends it to the device, where users can preview the interior and purchase items.

[1862] Step 10:

[1863] The terminal displays to the user an image of the final room layout, a list of suggested furnishings, and a link to purchase.

[1864] Input: Comprehensive data sent from the server.

[1865] Output: A room layout image shown to the user, a suggested furniture list, and a purchase link.

[1866] Specific operation: The user can view the layout image and suggested equipment list through the application on their device, and click on the provided link to go to the shopping site to purchase the items.

[1867] Through the above series of processing steps, the present invention provides a system that allows users to easily and efficiently realize their ideal room and purchase the necessary interior decorations and home appliances.

[1868] (Application example 1)

[1869] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1870] Traditionally, the process of selecting interior decor and home appliances to create the ideal room when moving into a new home has been extremely time-consuming and laborious. It is particularly difficult to visualize the layout of furniture and home appliances and select the appropriate items, and it is even more time-consuming to search for links to purchase each item one by one. It is also not easy to match the user's desired style with existing furniture. There is a need for a system that can solve these problems and efficiently support the creation of the ideal room.

[1871] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1872] In this invention, the server includes input means for inputting a room image, room dimension information, a list of necessary fixtures and electrical equipment, information on existing fixtures and electrical equipment, and a preferred style from the user, transmission means for transmitting data from the input means to the server, analysis means for receiving the data from the transmission means and performing image analysis and text data analysis, generation means for generating a room layout image using a generation AI based on the results of the analysis means, proposal means for generating a list of suggested fixtures and electrical equipment based on the layout generated by the generation means, link generation means for generating links to purchase pages for the fixtures and electrical equipment proposed by the proposal means, display means for transmitting and displaying the data generated by the link generation means to the user, and link acquisition means for acquiring purchase links for the suggested fixtures and electrical equipment. This allows users to easily generate their ideal room layout and purchase the necessary fixtures and electrical equipment.

[1873] A "user" is a user of the system who inputs information for selecting and arranging interior decor and home appliances for a new home.

[1874] "Room footage" refers to photos and videos of the room that users provide to the system.

[1875] "Room dimension information" is information about the physical size of a room, such as the length, width, and height of the room.

[1876] "Fixtures" refers to the interior and furniture of a room, and usually includes chairs, tables, shelves, etc.

[1877] "Electrical equipment" refers to electrical appliances and devices used in a room, such as a television or refrigerator.

[1878] "Input means" refers to a device or interface that allows a user to input room images, dimensional information, a list of fixtures and electrical equipment, etc. into the system.

[1879] "Transmission means" refers to a device or software for transmitting input data to a server.

[1880] "Analysis means" refers to functions and software for performing image analysis and text data analysis based on received data.

[1881] "Generation means" refers to functions and algorithms for generating a room layout image using generation AI based on the results of the analysis means.

[1882] The "proposal means" is a function or software that creates a list of proposed fixtures and electrical equipment based on the generated layout.

[1883] "Link generation means" refers to the functionality or software for generating links to the purchase pages of the proposed fixtures and electrical equipment.

[1884] "Display means" refers to a device or interface for displaying the generated data, suggested lists, and purchase links to the user.

[1885] "Link acquisition means" refers to the functionality or software for acquiring the purchase links for the proposed fixtures and electrical equipment.

[1886] "Style" refers to the interior design or theme that the user prefers, such as simple modern.

[1887] This invention is a system that assists users in purchasing interior decor and home appliances to create their ideal room when moving into a new home. This system uses generative AI to generate a room layout image based on user input and suggests specific fixtures and electrical equipment. The processing flow of the system's program and its specific operation are explained below.

[1888] System Overview

[1889] This system consists of a terminal and a server. The terminal sends the user's input information to the server and displays the final result to the user. The server analyzes the user's input information and uses generative AI to generate an optimal room layout image and proposal list.

[1890] Program processing flow and operation

[1891] 1. User Input

[1892] The user uses the terminal to input a picture of the room, the room's dimensions, a list of necessary fixtures and electrical equipment, information on the fixtures and electrical equipment they have, and their preferred style.

[1893] For example, you can input a video of the living room of your new home, the dimensions of the room (5m x 4m), the need for a TV stand and carpet, the sofa and table you already have, and your preferred style (simple modern).

[1894] 2. Data Transmission

[1895] The terminal transmits the collected user input data to the server.

[1896] 3. Data Receipt and Storage

[1897] The server receives the data sent from the device and stores it in a database for analysis.

[1898] 4. Image Analysis

[1899] The server retrieves images of the room from the database and performs image analysis, using image recognition algorithms to identify the layout of the room and the location and dimensions of existing furniture. For example, edge detection and object recognition algorithms are used to identify the contours of the room and the location of furniture.

[1900] 5. Text Data Analysis

[1901] The server analyzes the room dimensions, the list of required fixtures and electrical equipment, and the information on existing fixtures and electrical equipment, thereby understanding the size of the room and the required interior specifications.

[1902] 6. Style Analysis

[1903] The server performs an analysis to identify suitable fixtures and electrical equipment based on the user's preferred style (e.g., simple modern).

[1904] 7. Data integration and input information organization

[1905] The server integrates and organizes the results of image analysis, text data analysis, and style analysis, thereby clarifying the design conditions that are optimal for the user's wishes and the characteristics of the room.

[1906] 8. Layout Generation Using Generative AI

[1907] The server then calls a generation AI based on the integrated data to generate an optimal room layout image. This generation AI then uses the user's input information and analysis results to automatically arrange the layout and interior.

[1908] 9. Generate a list of fixtures and electrical equipment

[1909] The server creates a list of suggested furniture and electrical equipment for the user based on the room layout image generated by the AI. For example, it adds a simple, modern TV stand or carpet to the list.

[1910] 10. Generate shopping page links

[1911] The server generates a link to a purchasing page for each proposed fixture or electrical device, possibly using an online shopping API.

[1912] 11. Data Transmission

[1913] The server sends the device an image of the final room layout, a list of proposed fixtures and electrical equipment, and a link to purchase each item.

[1914] 12. Displaying Data

[1915] The terminal displays to the user an image of the final room layout, a list of suggested fixtures and electrical equipment, and a link to purchase.

[1916] 13. User Choices and Purchases

[1917] Users can check the information displayed on their device, select the items they like from the suggested fixtures and electrical equipment, and purchase them using the provided purchase link.

[1918] Hardware and software used

[1919] Hardware:

[1920] Smartphone (with camera function)

[1921] software:

[1922] Python, Pillow, and requests

[1923] Examples:

[1924] User A takes a video of their living room with their smartphone and enters the room dimensions (5m x 4m), a list of necessary furniture (TV stand, carpet), a list of furniture they already have (sofa, table), and their preferred style (simple modern) into an input form.

[1925] The app generates the best simple modern layout image and provides purchase links for the "Simple Modern TV Stand" and "Simple Modern Carpet."

[1926] Example prompts to input to a generative AI model:

[1927] Generate the optimal room layout image based on the user's room image, dimensions, required furniture list, existing furniture information, and preferred style.

[1928] Room video: <Video from user>

[1929] Room dimensions: 5m long, 4m wide

[1930] List of necessary furniture: TV stand, carpet

[1931] Current fixtures: sofa, table

[1932] Favorite style: Simple modern

[1933] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1934] Step 1:

[1935] The user uses a terminal to input a video of the room, information about the room's dimensions, a list of necessary fixtures and electrical equipment, information about the fixtures and electrical equipment they currently have, and their preferred style. A smartphone application is used as the input method. The data the user inputs might include, for example, a video of the living room in their new home, the dimensions of the room (5m long x 4m wide), the need for a TV stand and carpet, the sofa and table they currently have, and their preferred style being simple and modern. This input information is then saved as input data.

[1936] Step 2:

[1937] The terminal sends the collected user input data to the server. This transmission method uses the HTTP protocol via the Internet. The input data is converted into JSON format and sent to the server. This data transmission procedure passes data related to the user's needs to the server.

[1938] Step 3:

[1939] The server receives the data sent from the device and stores it in a database for analysis. The server, as the receiving means, verifies the accuracy of the data and stores it in the database if there are no problems. The stored data is necessary for subsequent analysis processing and serves as the basis for realizing the user's wishes.

[1940] Step 4:

[1941] The server retrieves images of the room from the database and performs image analysis. Image analysis determines the room layout and the location and dimensions of existing furniture. Specifically, edge detection and object recognition algorithms are used to identify the contours of the room and the location of furniture. The image of the room is used as input data, and data on the layout and dimensions of furniture is output.

[1942] Step 5:

[1943] The server analyzes the room's dimensions, a list of required fixtures and electrical equipment, and information on existing fixtures and electrical equipment. It then analyzes the text data to understand the room's size and required interior specifications. This information is used as input data, and the specific specifications needed for fixture placement and selection are output.

[1944] Step 6:

[1945] The server performs an analysis to identify suitable fixtures and electrical equipment based on the user's preferred style. Analysis of the style data allows for the selection of fixtures based on the user's preferred taste, such as simple modern. Style information is used as input data, and fixtures and electrical equipment that match the user's preferences are output.

[1946] Step 7:

[1947] The server integrates and organizes the results of image analysis, text data analysis, and style analysis. This clarifies the design conditions that are optimal for the user's wishes and the characteristics of the room. Multiple analysis results are integrated and the optimal layout conditions are output.

[1948] Step 8:

[1949] The server calls the generation AI based on the integrated data and ge...

Claims

1. An input means for the user to input a photo of the room, information on the size of the room, a list of necessary furniture and home appliances, information on the furniture and home appliances they currently own, and their preferred tastes; a transmitting means for transmitting data from the input means to a server; an analysis means for receiving data from the transmission means and performing image analysis and text data analysis; A generating means for generating a room layout image using a generating AI based on the result of the analyzing means; a proposal means for generating a list of furniture and home appliances to be proposed based on the layout generated by the generation means; a link generating means for generating a link to a purchase page for the furniture / home appliances suggested by the suggesting means; a display means for transmitting the data generated by the link generating means to a user and displaying the data; A system including:

2. 2. The system according to claim 1, wherein the link generating means generates the link taking into consideration the size and layout information of the proposed furniture and home appliances.

3. 2. The system according to claim 1, wherein the analyzing means performs the analysis based on the user's preferred taste.

Images