System

Abstract

To provide a system.SOLUTION: A system includes: means for acquiring information on interior design from a user; means for generating a prompt text to be input to a generative AI model on the basis of the acquired interior design information; means for inputting the generated prompt text to the generative AI model to obtain an interior layout proposal from the generative AI model; and means for outputting the acquired layout proposal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] It is difficult to plan a room layout on your own, and it is also difficult to materialize your image. Furthermore, there are no systems that suggest layouts that match the actual room environment. There are also no systems that make optimal suggestions based on behavioral economics regarding the placement of products and shelves. [Means for solving the problem]

[0005] Using a generative AI that has learned various room layouts, the system generates and proposes layouts suited to the actual environment based on chat-based image input from the user and data on the actual environment of the target room.The system also supports specific product selection by outputting product information for furniture and home appliances to be placed.Furthermore, by including an extension function that suggests product / shelf placement methods based on behavioral economics, it supports optimal product placement in commercial facilities, etc. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one example of the present invention, a user inputs an image of their room (e.g., "a living room with a simple and calm atmosphere") through a chat-based interface. This input is analyzed by the generative AI and combined with previously learned room layout data to generate new layout proposals.

[0029] "Example 2"

[0030] In addition, users input real-world environmental data (e.g., data captured by LiDAR, images, videos, etc.) into the system. This data is analyzed by generative AI, and a layout proposal tailored to the real environment is generated.

[0031] "Example 3"

[0032] The generated layout proposal includes specific product information (e.g., manufacturer name, product name, price, etc.) for the furniture and home appliances to be placed, allowing users to use this information as a reference for specific product selection.

[0033] "Example 4"

[0034] Furthermore, an example embodiment of the present invention includes an extension function that makes optimal suggestions based on behavioral economics regarding the placement of products and product shelves in commercial facilities, etc. This function makes it possible to propose optimal product placement that encourages customer purchasing behavior.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: The user inputs an image of their room (e.g., "a living room with a simple and calm atmosphere") through a chat-based interface.

[0038] Step 2: This input is parsed by the generative AI.

[0039] Step 3: The generation AI combines the learned room layout data with input from the user to generate new layout proposals.

[0040] "Example 2"

[0041] Step 1: The user inputs real-world environmental data (e.g., data captured by LiDAR, images, videos, etc.) of the target room into the system.

[0042] Step 2: This data is analyzed by the generative AI.

[0043] Step 3: The generation AI combines the real-world data with the learned room layout data to generate layout proposals tailored to the real-world environment.

[0044] "Example 3"

[0045] Step 1: The generated layout proposal includes specific product information (e.g., manufacturer name, product name, price, etc.) of the furniture and home appliances to be placed.

[0046] Step 2: The user uses this product information to make a specific product selection.

[0047] "Example 4"

[0048] Step 1: The system will include an extension that makes optimal suggestions based on behavioral economics regarding product and shelf placement in commercial facilities, etc.

[0049] Step 2: This feature suggests optimal product placement to encourage customer purchasing behavior.

[0050] Example 1

[0051] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0052] Conventional room layout suggestion systems have difficulty proposing layouts that reflect the user's specific image, and making suggestions that are appropriate for the actual environment. Furthermore, they lack specific furniture and appliance suggestions based on the user's desired atmosphere and style.

[0053] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0054] In this invention, the server includes a chat-based image input means for a user, a means for transmitting the input image to the server, a means for the server to pass the received image to a generative AI model, a means for the generative AI model to analyze the input and combine it with learned layout data to generate a new layout proposal, a means for the server to transmit the generated layout proposal to a terminal, and a means for the terminal to display the layout proposal to the user. This makes it possible to propose a layout based on the user's specific image, and also to propose specific furniture and home appliances suitable for the actual environment.

[0055] The "chat-based image input means from the user" is a means for providing an interface for the user to input an image of the room in text format.

[0056] The "means for transmitting the input image to the server" is a communication means for transmitting the text data input by the user to the server.

[0057] The "means for passing the image received by the server to the generative AI model" is a processing means for passing the text data received by the server to the generative AI model.

[0058] "The means by which the generative AI model analyzes input and combines it with previously learned layout data to generate new layout proposals" refers to the means by which the generative AI model analyzes user input and combines it with previously learned data to generate new layout proposals.

[0059] The "means by which the server transmits the generated layout proposal to the terminal" refers to a communication means by which the server transmits the layout proposal received from the generative AI model to the terminal.

[0060] The "means for the terminal to display the layout proposal to the user" refers to a display means for the terminal to display the layout proposal received from the server to the user.

[0061] This invention is a system in which a user inputs an image of a room through a chat-based interface, and a generative AI model generates new layout proposals based on that image. Specific embodiments of this system are described below.

[0062] Users access the chat interface using a smartphone or PC browser, which allows them to input a text image of the room they want to live in. For example, they might enter "a living room with a simple and relaxing atmosphere."

[0063] The device sends the text data entered by the user to the server as an HTTP POST request. This request includes the user's input text. The server receives the HTTP POST request, extracts the user's input text from the request body, and then creates an API request to pass this text to the generative AI model and sends it to the generative AI model.

[0064] The generative AI model uses, for example, a model using natural language processing technology (e.g., GPT-4 (registered trademark)). This model analyzes the user's input text and combines it with previously learned room layout data to generate new layout proposals. Based on the analysis results, it can generate layout proposals such as the following:

[0065] The wall color is light gray

[0066] The sofa has a simple design and is beige in color.

[0067] The coffee table is made of wood and has a natural color.

[0068] The lighting makes extensive use of indirect lighting to provide soft light.

[0069] The server sends the layout proposal received from the generative AI model to the device as an HTTP response. This response contains the generated layout proposal. The device parses the HTTP response received from the server and extracts the layout proposal. The device then displays the proposal on the chat interface. The user can review the proposed layout and re-enter it if necessary.

[0070] For example, if a user inputs "I want to create a simple and calm living room," a generative AI model might generate the following layout suggestions:

[0071] The wall color is light gray

[0072] The sofa has a simple design and is beige.

[0073] The coffee table is made of wood and has a natural color.

[0074] The lighting makes extensive use of indirect lighting to provide soft light.

[0075] Example prompt sentence:

[0076] User: I want to create a simple and calm living room.

[0077] Server: Analyzing. Please wait a moment.

[0078] Generative AI model: The wall color is pale gray, the sofa is simple and beige, the coffee table is made of wood and has a natural color, and the lighting is indirect, providing soft light.

[0079] Server: The proposed layout is as follows:

[0080] Terminal: The wall color is pale gray, the sofa is simply beige, the coffee table is made of wood and has a natural color, and the lighting is indirect, providing soft light.

[0081] In this way, users can easily receive room layout suggestions through a chat-based interface.

[0082] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0083] Step 1:

[0084] The user inputs an image of the room into the chat interface.

[0085] The user opens a browser on their smartphone or PC, accesses the chat interface, types in, for example, "a living room with a simple and calm atmosphere," and clicks the send button. The input data is in text format.

[0086] Step 2:

[0087] The device sends input to the server.

[0088] The terminal sends the text data entered by the user to the server as an HTTP POST request. This request contains the user's input text. The input data is in text format, and the output data is an HTTP request.

[0089] Step 3:

[0090] The server receives the input and passes it to the generative AI model.

[0091] The server receives the HTTP POST request, extracts the user's input text from the request body, creates an API request to pass this text to the generative AI model, and sends it to the model. The input data is in text format, and the output data is an API request.

[0092] Step 4:

[0093] The generative AI model analyzes the input and combines it with learned layout data to generate new layout proposals.

[0094] The generative AI model receives API requests and analyzes the user's input text. Based on the analysis results, it combines them with the learned room layout data to generate new layout proposals. For example, it generates proposals such as "The wall color should be light gray, the sofa should have a simple beige design, the coffee table should be made of wood and have a natural color, and the lighting should be indirect, providing soft light." The input data is in text format, and the output data is the layout proposal.

[0095] Step 5:

[0096] The server receives the generated layout proposal and transmits it to the terminal.

[0097] The server sends the layout proposal received from the generative AI model to the terminal as an HTTP response. This response contains the generated layout proposal. The input data is the layout proposal, and the output data is the HTTP response.

[0098] Step 6:

[0099] The terminal displays the layout proposal to the user.

[0100] The device analyzes the HTTP response received from the server and extracts layout suggestions. It then displays these suggestions on the chat interface. The user can review the proposed layout and re-enter it if necessary. The input data is the HTTP response, and the output data is the displayed layout suggestions.

[0101] (Application example 1)

[0102] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] Conventional interior design support systems have difficulty generating layout proposals that specifically reflect the user's desired image of the space. Furthermore, there is a lack of tools to efficiently support the interior design of brick-and-mortar stores, and there has been a demand for a system that allows users to easily receive layout proposals using their smartphones.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0105] In this invention, the server includes a generative AI means that has learned the layouts of various spaces, a chat-based image input means from a user, a means for inputting real-world environmental data of the target space, a means for generating / proposing a layout that suits the real environment, a means for outputting product information of items to be placed, a means for supporting the interior design of a physical store, a means including an application to be installed on a smartphone, and a means for generating a layout proposal using the generative AI model. This makes it possible to generate a layout proposal that specifically reflects the image of the space desired by the user, and to efficiently support the interior design of a physical store.

[0106] "Generative AI means" is an artificial intelligence technology that learns the layouts of various spaces and generates new layout proposals based on user input.

[0107] The "chat-based image input means" is an interface that allows users to input an image of the desired space in text format.

[0108] "Means for inputting real-world data" refers to a means for acquiring the current state of the target space as data and inputting it into the system.

[0109] "Means for generating / proposing layouts" refers to means for proposing new layouts to users based on data analyzed by the generation AI means.

[0110] The "means for outputting product information of an item" is a means for providing the user with detailed information about the items to be arranged in the layout.

[0111] "Means to support the interior design of physical stores" refers to means that provide support for efficiently designing the space of physical stores.

[0112] "Means including an application installed on a smartphone" means means that allows a user to use the system through an application that runs on a smartphone.

[0113] "Means for generating layout proposals using a generative AI model" means means for generating layout proposals based on user input using a generative AI model.

[0114] A system for implementing this invention is configured as follows: First, a user uses an application installed on a smartphone to input an image of a desired space into a chat-based interface. This input is analyzed by a generative AI means. The generative AI means has learned the layouts of various spaces and generates new layout proposals based on the user's input.

[0115] The server receives image input from the user and acquires current spatial data using a means of inputting real-world environmental data for the target space. This data can be collected, for example, using a smartphone camera or sensors. The server then uses generative AI means to generate and propose a layout that fits the real-world environment.

[0116] The generated layout proposal is displayed to the user through a smartphone application, and product information for the items to be placed is also displayed at the same time. This allows the user to purchase the necessary items based on the proposed layout.

[0117] As a concrete example, if the user requests a "cafe with a simple and relaxing atmosphere," the following prompt sentence is generated:

[0118] Example prompt sentence:

[0119] The image of the store desired by users: A simple and calm cafe

[0120] Generate a new layout proposal.

[0121] By inputting this prompt into a generative AI model, new store layout proposals that match the user's wishes can be obtained. For example, the GPT-3 model from OpenAI (registered trademark) is used as the generative AI model. This model analyzes the user's input and generates optimal layout proposals based on the data it has already learned.

[0122] This system allows users to easily receive interior design suggestions using their smartphones, making it possible to efficiently design the interiors of physical stores.

[0123] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0124] Step 1:

[0125] Users launch an application installed on their smartphone and input an image of the space they want into a chat-based interface. The input data is in text format and includes a specific image, such as "a cafe with a simple and relaxing atmosphere." This input data is then passed on to the next step of processing.

[0126] Step 2:

[0127] The terminal receives image input from the user and sends it to the generation AI means. The generation AI means analyzes the input text data and generates a prompt sentence. This prompt sentence is converted into a format that can be input to the generation AI model. For example, the prompt sentence might be, "The image of the store desired by the user: A cafe with a simple and relaxing atmosphere. Please generate a new layout proposal."

[0128] Step 3:

[0129] The server inputs the generated prompt sentence into a generative AI model (e.g., OpenAI's GPT-3 model), which analyzes the prompt sentence and generates new layout proposals based on the learned data. These layout proposals are then passed to the next step.

[0130] Step 4:

[0131] The server receives the generated layout proposal and combines it with real-world data of the target space. The real-world data is acquired using the smartphone's camera and sensors. This data includes the space's dimensions and the layout of existing furniture. The server integrates this data and generates an optimal layout proposal.

[0132] Step 5:

[0133] The server sends the final layout proposal to the smartphone application. The device displays this proposal to the user. The user can review the proposed layout and make any necessary modifications. Product information for the items to be placed is also displayed at the same time, allowing the user to purchase the items they need.

[0134] Step 6:

[0135] If the user is satisfied with the proposed layout, they finalize it. The device then sends this information to the server, which then performs subsequent processes such as saving the layout and ordering items. This allows the user to efficiently realize their interior design.

[0136] Example 2

[0137] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0138] Conventional room layout proposal systems are limited in the types and formats of data that users can input, making it difficult to provide specific proposals that are suited to the actual environment. Furthermore, the generated layout proposals often do not meet the user's expectations, resulting in the need to make re-proposals, which is time-consuming. Furthermore, it is difficult to confirm whether the proposed layout is actually possible.

[0139] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0140] In this invention, the server includes a means for preprocessing received data, a means for inputting the preprocessed data into a generative AI model, and a means for returning the generated layout proposal to the user. This makes it possible to accommodate a variety of data formats input by users and to provide specific layout proposals that are suited to the actual environment. Furthermore, even if the generated layout proposal does not meet the user's expectations, a new proposal can be quickly made. Furthermore, it becomes easy to confirm whether the proposed layout can actually be arranged.

[0141] "Generative AI means" refers to artificial intelligence technology that learns various room layouts and generates optimal layout suggestions based on input data.

[0142] "Chat-based image input means" refers to an interface that allows users to input images and requests in text format.

[0143] "Means for inputting real-world data" refers to an interface that allows users to upload real-world data, such as LiDAR data, images, and videos of the target room, to the system.

[0144] "Means for generating / proposing layouts" refers to the function in which the AI ​​generating means generates the optimal room layout based on the input real-world environment data and proposes it to the user.

[0145] "Means for outputting product information" refers to a function that provides the user with information about the furniture and home appliances to be placed based on the generated layout.

[0146] "Means for preprocessing received data" refers to the process of analyzing data such as LiDAR data, images, and videos received from users and extracting necessary information.

[0147] "Means for inputting preprocessed data into a generative AI model" refers to the process of inputting preprocessed data into a generative AI means to generate optimal layout proposals.

[0148] "Means for returning the generated layout proposals to the user" refers to an interface for providing the layout proposals generated by the generating AI means to the user.

[0149] This invention is a system that allows a user to input real-world environment data of a target room and generates an optimal layout proposal using a generative AI model. Specific embodiments of this system are described below.

[0150] First, the user inputs real-world data from the target room into the system. This data includes 3D data acquired using a LiDAR scanner, images and videos taken with a smartphone, etc. This data is uploaded from the user's device through the system interface.

[0151] The server receives data uploaded by users. The received data may be in the form of LiDAR data, images, videos, etc. The server first preprocesses this data. Specifically, it converts the LiDAR data into a 3D model and extracts room features from the images and videos. For example, it extracts information such as wall color, furniture placement, and window positions.

[0152] The server then inputs the preprocessed data into a generative AI model, which uses advanced AI techniques such as OpenAI's GPT-4 or DALL-E, to generate an optimal layout based on the input data.

[0153] The server generates specific layout proposals based on the analysis results obtained from the generative AI model. These proposals include furniture placement, design ideas, color combinations, and more. The generated layout proposals are returned to the user. The user can review the proposals through the system interface and request revisions or re-proposals as needed.

[0154] As a concrete example, consider a user who wants to improve the layout of their living room. The user uses a LiDAR scanner to obtain 3D data of the living room and takes photos and videos of the room with their smartphone. These data are then uploaded to the system. The server first analyzes the LiDAR data and generates a 3D model of the room. It then extracts information such as furniture placement and wall color from the images and videos. Based on this data, a generative AI model generates optimal layout suggestions.

[0155] An example of a specific prompt is, "Please analyze the 3D data and images of the living room and suggest optimal furniture placement and design ideas." In this way, users can receive specific layout suggestions from the generative AI model based on the real-world environment data of the room.

[0156] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0157] Step 1:

[0158] The user inputs real-world data

[0159] The user inputs real-world data of the target room into the system. Specifically, the user acquires 3D data of the room using a LiDAR scanner, takes photos and videos of the room using a smartphone, and uploads this data through the system interface. The input data includes LiDAR data, images, and videos.

[0160] Step 2:

[0161] The server receives the data

[0162] The server receives the data uploaded by the user, which may be in the form of LiDAR data, images, videos, etc. The server temporarily stores this data and prepares it for the next processing step.

[0163] Step 3:

[0164] The server preprocesses the data

[0165] The server preprocesses the received data. Specifically, it converts the LiDAR data into a 3D model and extracts room features from images and videos. For example, it extracts information such as wall color, furniture placement, and window position. The input data are LiDAR data, images, and videos, and the output data are the 3D model and room feature information.

[0166] Step 4:

[0167] The server analyzes the data using a generative AI model

[0168] The server inputs the preprocessed data into a generative AI model, which uses advanced AI technology, for example. The generative AI model generates an optimal layout based on the input data. The input data is a 3D model and room feature information, and the output data is a layout proposal.

[0169] Step 5:

[0170] The server generates a layout proposal

[0171] The server generates specific layout proposals based on the analysis results obtained from the generative AI model. These proposals include furniture placement, design ideas, color combinations, etc. The input data is the analysis results of the generative AI model, and the output data is the specific layout proposals.

[0172] Step 6:

[0173] The server returns the proposal to the user

[0174] The server returns the generated layout proposal to the user. The user can review the proposal through the system interface and request corrections or resubmissions as necessary. The input data is the specific layout proposal, and the output data is the proposal information to the user.

[0175] (Application example 2)

[0176] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] Conventional store layout optimization relies heavily on experience and intuition, making it difficult to achieve efficient layouts. In addition, there has been a lack of layout proposals that utilize real-world data, making it difficult to achieve optimal layouts that are in line with the actual store environment.

[0178] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout that suits the real environment, a means for outputting product information of the furniture / appliances to be placed, a means for capturing real-world environment data of the store, a means for analyzing the data using a generation AI model and generating an optimal layout proposal, and a means for displaying the generated layout proposal. This enables efficient layout proposals that are suited to the actual store environment.

[0179] A "generative AI method" is an artificial intelligence method that learns various room layouts and generates the optimal layout based on input data.

[0180] A "chat-based image input means" is a means for users to input images and requests in text format.

[0181] "Means for inputting actual environment data" refers to means for inputting actual environment data of the target room or store into the system.

[0182] The "means for generating / proposing layouts" is a means for generating optimal layouts based on input real-world data and proposing them to the user.

[0183] The "means for outputting product information" is a means for providing detailed product information of the furniture and home appliances to be placed to the user.

[0184] "Means for capturing real-world store environment data" refers to means for obtaining real-world environment data such as LiDAR data, images, and videos within the store.

[0185] "Means for analyzing data using a generative AI model" refers to a means for inputting acquired real-world data into a generative AI model and performing analysis.

[0186] The "means for displaying a layout proposal" is a means for visually displaying the generated layout proposal to the user.

[0187] A system for implementing this invention is configured as follows. First, a user captures real-world data in a store using a smartphone. Specifically, the user acquires 3D data using a LiDAR sensor and takes images and videos using the smartphone's camera. This data is then sent to a server via an application on the smartphone.

[0188] The server uses a generative AI model to analyze the received real-world data. The generative AI model has previously learned about various room layouts and generates optimal layout proposals based on the input data. Specifically, the process is carried out as follows:

[0189] 1. Data capture:

[0190] Users use their smartphones to capture LiDAR data and image data inside the store.

[0191] This data is sent to a server via a smartphone application.

[0192] 2. Data Analysis:

[0193] The server inputs the received LiDAR data and image data into the generative AI model.

[0194] The generative AI model generates optimal layout suggestions based on the input data.

[0195] 3. Generate layout proposals:

[0196] The generative AI model analyzes real-world data and generates optimal layout proposals.

[0197] The generated layout proposal is displayed on the user's smartphone.

[0198] The hardware used includes a smartphone, LiDAR sensor, and smartphone camera, while the software used includes a generative AI model (e.g., GPT-3), an application for data capture, and a server for data analysis.

[0199] As a concrete example, the following prompt sentence is input to the generative AI model:

[0200] Example prompt sentence:

[0201] LiDAR data: [0.5, 1.2, 0.3, ...], Image data: [image_data_as_base64_string]

[0202] By inputting this prompt into a generative AI model, the model generates suggestions for optimizing the store layout, such as "rearrange product placement to improve customer flow" or "place specific products in prominent locations."

[0203] In this way, it becomes possible to propose an efficient layout that is suited to the actual store environment.

[0204] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0205] Step 1:

[0206] Users use their smartphones to capture real-world data in the store. Specifically, they acquire 3D data using a LiDAR sensor and take images and videos using the smartphone's camera. The input data is the LiDAR data and image data, which are then saved in the smartphone's application.

[0207] Step 2:

[0208] The user sends the captured real-world data to the server through a smartphone application. The input data is LiDAR data and image data, which are uploaded to the server via the network. The output is the real-world data stored on the server.

[0209] Step 3:

[0210] The server uses a generative AI model to analyze the received real-world data. Specifically, it inputs LiDAR data and image data into the generative AI model. The input data are LiDAR data and image data, and the generative AI model analyzes them to generate an optimal layout proposal. The output is a layout proposal.

[0211] Step 4:

[0212] The server sends the generated layout proposal to the user's smartphone. The input data is the layout proposal from the generative AI model, which is sent to the user's smartphone via the network. The output is the layout proposal displayed on the user's smartphone.

[0213] Step 5:

[0214] The user checks the generated layout proposal through a smartphone application. The input data is the layout proposal sent from the server, which is displayed on the smartphone screen. The output is the layout proposal that the user checks.

[0215] In this way, it becomes possible to propose an efficient layout that is suited to the actual store environment.

[0216] Example 3

[0217] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0218] Conventional layout proposal systems make it difficult for users to obtain specific product information and to determine whether the proposed layout is suitable for the actual environment. In addition, in commercial facilities, they are unable to propose optimal product placement to encourage customer purchasing behavior.

[0219] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout suited to the real environment, a means for outputting product information of the furniture / appliances to be arranged, a means for acquiring product information from a database based on the generated layout proposal, a means for incorporating the acquired product information into the layout proposal, and a means for returning the final layout proposal to the user. This allows the user to obtain a layout proposal including specific product information, making it easy to determine a layout suitable for the actual environment. Furthermore, in commercial facilities, optimal product placement can be proposed to promote customer purchasing behavior.

[0220] "Generative AI means" refers to artificial intelligence that learns various room layouts and generates optimal layout suggestions based on the user's requirements.

[0221] "Chat-based image input means" refers to an interface that allows users to input layout preferences and conditions in text format.

[0222] "Means for inputting real-world data" refers to a means for inputting real-world environmental data, such as the dimensions and shape of the target room and the existing furniture layout, into the system.

[0223] "Means for generating / proposing layouts" refers to means for generating an optimal room layout based on input data and proposing it to the user.

[0224] "Means for outputting product information" refers to means for providing the user with specific product information (manufacturer name, product name, price, etc.) of the furniture and home appliances included in the generated layout.

[0225] The "means for acquiring product information from a database" refers to a means for searching and acquiring necessary product information from a database based on the generated layout proposal.

[0226] The "means for incorporating product information into a layout proposal" refers to a means for integrating acquired product information into a generated layout proposal and providing it to a user.

[0227] The "means for returning a final layout proposal to the user" refers to a means for transmitting the final generated layout proposal to the user.

[0228] This invention is a system for allowing a user to receive room layout proposals, and generates layout proposals including specific product information using a generative AI model. A specific embodiment of this system is described below.

[0229] Server Roles

[0230] The server receives requests from users and generates layout proposals using generative AI models. The server uses the following hardware and software:

[0231] Hardware: High-performance server machine (e.g., CPU, memory, storage)

[0232] Software: Generative AI model, database management system, API server

[0233] The server receives a request from the user, generates and inputs a prompt to the generative AI model, analyzes the generated layout proposal, and retrieves the necessary product information from the database. The retrieved product information is then incorporated into the layout proposal and sent back to the user.

[0234] Device Role

[0235] The device sends user-entered requests to the server and displays layout suggestions returned by the server. The device uses the following hardware and software:

[0236] Hardware: PCs, smartphones, tablets

[0237] Software: Web browser, dedicated application

[0238] The terminal provides an interface for the user to input requests, transmits the requests to the server, receives layout proposals returned from the server, and displays them to the user.

[0239] User Roles

[0240] Users can request a layout proposal through their device, review the returned proposal, and use the specific product information included in the proposal to help them select furniture and home appliances.

[0241] Specific examples

[0242] Here are some example prompts to input to the generative AI model:

[0243] "Please suggest a living room layout that meets the following criteria:

[0244] Includes sofa, TV and coffee table

[0245] Modern design

[0246] The budget is under 200,000 yen.

[0247] When this prompt is sent to the server, the generative AI model generates a living room layout proposal and returns it, including specific product information (e.g., manufacturer name, product name, price, etc.). The user can use this information to select furniture and appliances.

[0248] In this way, the user can obtain layout proposals including specific product information, and can easily determine a layout suitable for the actual environment. Also, in commercial facilities, the system can propose optimal product placement to promote customer purchasing behavior. The flow of the specification process in the third embodiment will be described with reference to FIG. 15.

[0249] Step 1: Receiving a user request

[0250] The server receives a layout proposal request from the user. Specifically, it receives an HTTP request and analyzes the request content. The input is the request data sent by the user from the terminal, and the output is the analyzed request content.

[0251] Step 2: Generate a prompt statement

[0252] The server generates a prompt sentence to be input to the generative AI model based on the user's request. Specifically, it uses a template to construct a prompt sentence based on the user's input data. The input is the parsed request content, and the output is the generated prompt sentence.

[0253] Step 3: Input to the generative AI model

[0254] The server inputs the generated prompt sentence into the generative AI model and generates a layout proposal. Specifically, it calls the API of the generative AI model and sends the prompt sentence. The input is the generated prompt sentence, and the output is the layout proposal returned from the generative AI model.

[0255] Step 4: Get product information

[0256] The server retrieves the necessary product information from a database based on the generated layout proposal. Specifically, it executes a database query to retrieve information such as the manufacturer, product name, and price of the furniture or home appliance. The input is the generated layout proposal, and the output is the retrieved product information.

[0257] Step 5: Assembling the layout proposal

[0258] The server incorporates the acquired product information into the layout proposal to create the final proposal. Specifically, it integrates the generated layout proposal with the product information and constructs data to be sent back to the user. The input is the acquired product information, and the output is the final layout proposal.

[0259] Step 6: Return to user

[0260] The server returns the final layout proposal to the user. Specifically, it generates an HTTP response and sends it to the user's device. The input is the final layout proposal, and the output is the layout proposal sent to the user's device.

[0261] Step 7: Providing a Request Interface

[0262] The terminal provides an interface for the user to request a layout proposal. Specifically, it displays a form and input fields to allow the user to enter conditions. The input is the user's operation, and the output is the request data entered by the user.

[0263] Step 8: Submitting the Request

[0264] The terminal sends the request entered by the user to the server. Specifically, when the user submits a form, an HTTP request is sent to the server. The input is the request data entered by the user, and the output is the request sent to the server.

[0265] Step 9: Receiving the Response

[0266] The terminal receives the layout proposal returned from the server. Specifically, it receives the HTTP response and analyzes the data. The input is the layout proposal returned from the server, and the output is the analyzed layout proposal.

[0267] Step 10: View layout suggestions

[0268] The terminal displays the received layout proposal to the user. Specifically, the layout proposal and product information are displayed on the screen so that the user can confirm them. The input is the analyzed layout proposal, and the output is the layout proposal displayed to the user.

[0269] Step 11: Enter your request

[0270] The user requests a layout proposal through the terminal interface. Specifically, the user enters the conditions into a form and clicks the submit button. The input is the user's desired conditions, and the output is the request data entered into the terminal.

[0271] Step 12: Review the proposal

[0272] The user checks the layout proposal returned from the server. Specifically, the user views the layout proposal and product information displayed on the terminal. The input is the layout proposal displayed on the terminal, and the output is the user's confirmation result.

[0273] Step 13: Product Selection Recommendations

[0274] The user selects furniture or home appliances based on the specific product information included in the proposal. Specifically, the user decides which furniture or home appliance to purchase based on the proposed product information. The input is the proposed product information, and the output is the user's selection.

[0275] (Application example 3)

[0276] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0277] Product placement in traditional stores often relied on experience and intuition, making it difficult to find the optimal placement. Furthermore, there was a lack of means to provide specific product information, leaving users with a lack of reference information to use when making product selections. Furthermore, it was not possible to incorporate a scientific approach based on behavioral economics, making it difficult to effectively promote customer purchasing behavior.

[0278] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0279] In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from the user, a means for inputting actual environment data of the target room, a means for generating / proposing a layout that suits the actual environment, a means for outputting product information for the furniture / appliances to be arranged, a means for proposing product arrangements based on behavioral economics, a means for providing a smartphone application for optimizing product arrangements in a store, and a means for providing specific product information (manufacturer name, product name, price, etc.).By scientifically optimizing product arrangements in a store and providing specific product information, it is possible to enrich reference information for users when making product selections and effectively promote customer purchasing behavior.

[0280] A "generative AI means" is an artificial intelligence means that learns various room layouts and generates the optimal layout based on the user's requirements.

[0281] A "chat-based image input means" is a means for users to input images and requests in a chat format.

[0282] The "means for inputting actual environment data" refers to a means for inputting actual environment data of the target room or store.

[0283] The "means for generating / proposing a layout" is a means for generating and proposing an optimal layout based on input data.

[0284] The "means for outputting furniture / home appliance product information" is a means for outputting specific product information (manufacturer name, product name, price, etc.) of the furniture or home appliance to be arranged.

[0285] "Means for proposing product placement based on behavioral economics" refers to means for proposing optimal product placement that promotes customer purchasing behavior based on the theory of behavioral economics.

[0286] The "means for providing a smartphone application" is a means for providing an application on a smartphone for optimizing product placement in a store.

[0287] The "means for providing specific product information" refers to a means for providing specific product information such as the manufacturer name, product name, price, etc. to the user.

[0288] To implement the present invention, the following system configuration and processing procedures are required.

[0289] First, the server is equipped with a generation AI means that has learned various room layouts. This generation AI means is an artificial intelligence that generates the optimal layout based on past layout data. Users can input their desired layout and image using a chat-based image input means. This communicates the user's specific requests to the system.

[0290] Next, the server is equipped with a means for inputting the actual environment data of the target room or store. This means inputs data such as the dimensions and layout of the actual room or store. Based on this data, the generation AI means generates and proposes a layout that suits the actual environment.

[0291] Furthermore, the server is equipped with a means for outputting product information for the furniture and home appliances to be placed. This means allows the user to check specific product information (manufacturer name, product name, price, etc.), which can be used as a reference for the user in selecting specific products.

[0292] The server also has a means for proposing product placement based on behavioral economics. This allows it to propose optimal product placement that encourages customer purchasing behavior, thereby scientifically optimizing product placement within the store.

[0293] Furthermore, the server includes a means for providing a smartphone application for optimizing product placement in a store. By using this application, the store manager or staff can optimize product placement using their smartphones.

[0294] The server also has a means for providing specific product information, which allows the user to obtain specific product information such as the manufacturer's name, product name, price, etc.

[0295] The hardware used is a smartphone, and the software uses Python and the OpenAI API. Data processing involves formatting store data (product list, current layout information) in JSON format, and then using a generative AI model to generate the optimal product placement.

[0296] As a concrete example, by inputting the following prompt sentence into the generative AI model, optimal product placement suggestions can be obtained.

[0297] Example prompt sentence:

[0298] Store data: {"Store name": "ABC Store", "Product list": [{"Manufacturer name": "Manufacturer A", "Product name": "Product A", "Price": 1000}, {"Manufacturer name": "Manufacturer B", "Product name": "Product B", "Price": 2000}], "Store layout": "Current layout information"}

[0299] Please suggest optimal product placement based on behavioral economics.

[0300] In this way, the server, terminal, and user can work together to achieve optimal product placement.

[0301] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0302] Step 1:

[0303] The user starts the application on their smartphone and inputs their desired layout and image using a chat-based image input means.

[0304] Input: The layout and image desired by the user

[0305] Output: User's desired layout and image data

[0306] Step 2:

[0307] The server receives the image data sent from the user and inputs data such as the dimensions and layout of the actual room or store using a means for inputting real environment data of the target room or store.

[0308] Input: User image data, real-world data

[0309] Output: Integrated layout data

[0310] Step 3:

[0311] The server uses a generation AI means to generate an optimal layout based on the integrated layout data. The generation AI means learns from past layout data and proposes an optimal layout taking into account the user's requests and actual environment data.

[0312] Input: Integrated layout data

[0313] Output: Optimal layout proposal

[0314] Step 4:

[0315] The server provides the user with specific product information (manufacturer name, product name, price, etc.) for the furniture and home appliances to be arranged based on the generated layout proposal, using a means for outputting product information.

[0316] Input: Optimal layout proposal

[0317] Output: Specific product information

[0318] Step 5:

[0319] The server uses a method based on behavioral economics to suggest optimal product placements that encourage customer purchasing behavior, thereby scientifically optimizing product placements within the store.

[0320] Input: Optimal layout proposals, behavioral economics data

[0321] Output: Optimal product placement proposal

[0322] Step 6:

[0323] The server notifies the store manager and staff of the optimal product placement using a means for providing a smartphone application for optimizing product placement within the store.

[0324] Input: Optimal product placement proposal

[0325] Output: Notification on smartphone application

[0326] Step 7:

[0327] Users can use the smartphone application to check specific product information and use it as a reference for product selection.

[0328] Input: Specific product information

[0329] Output: User selected product information

[0330] In this way, the server, terminal, and user can work together to achieve optimal product placement.

[0331] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0332] "Example 1"

[0333] One embodiment of the present invention provides a system incorporating an emotion engine. This system recognizes a user's emotions and adjusts room layout suggestions based on those emotions. Specifically, if a user inputs that they desire a "relaxed living room," the emotion engine recognizes the user's current emotional state (e.g., relaxed, tense, etc.). It then adjusts the layout suggestions to match the user's emotional state, for example, by suggesting furniture with more subdued colors if the user is relaxed, or by suggesting furniture arrangements that create a relaxing effect if the user is tense.

[0334] "Example 2"

[0335] The emotion engine is also used to suggest product choices. For example, if a user inputs "I'm looking for a new TV," the emotion engine will recognize the user's current emotional state and suggest product choices based on that emotional state. For example, if the user is feeling happy, it will suggest a large-screen, high-end TV, and if the user is feeling sad, it will suggest a cost-effective TV.

[0336] "Example 3"

[0337] Furthermore, the emotion engine learns changes in the user's emotions and optimizes layout suggestions and product selection suggestions based on the results of that learning. For example, if a user feels a strong sense of joy when looking at a certain product, the engine will prioritize suggestions for that product or products in the same category. The engine learns how users respond to emotions and makes suggestions that reflect this.

[0338] "Example 4"

[0339] The emotion engine also analyzes users' emotional responses and makes product placement suggestions based on the results. For example, if a user expresses joy at the placement of a certain product shelf, the engine will apply that same placement to other product shelves.

[0340] The processing flow of each embodiment will be described below.

[0341] "Example 1"

[0342] Step 1: The user inputs their desired room layout into the system.

[0343] Step 2: The emotion engine recognizes the user's current emotional state.

[0344] Step 3: Adjust the layout proposal based on the emotional state recognized by the emotion engine.

[0345] "Example 2"

[0346] Step 1: The user enters their product selection preferences into the system.

[0347] Step 2: The emotion engine recognizes the user's current emotional state.

[0348] Step 3: Tailor product selection suggestions based on the emotional state recognized by the emotion engine.

[0349] "Example 3"

[0350] Step 1: The emotion engine learns the user's emotional changes.

[0351] Step 2: Based on the results learned by the emotion engine, layout suggestions and product selection suggestions are optimized.

[0352] "Example 4"

[0353] Step 1: The emotion engine analyzes the user's emotional response.

[0354] Step 2: Based on the results of the emotion engine's analysis, product placement suggestions are made.

[0355] Example 1

[0356] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] Conventional room layout suggestion systems make uniform suggestions without considering the user's emotional state, making it difficult to provide a layout that suits the user's current psychological state. In addition, layouts generated based on user input may not match the actual environment or emotional state, resulting in a decrease in user satisfaction.

[0358] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout suited to the real environment, a means for outputting product information of furniture / appliances to be placed, an emotion engine means for recognizing the emotional state of the user, and a means for adjusting the layout proposal based on the emotional state. This makes it possible to propose a layout suited to the emotional state of the user, thereby improving user satisfaction.

[0359] "Generative AI means" is an artificial intelligence technology that learns various room layouts and generates new layout suggestions based on user input.

[0360] The "chat-based image input means" is an interface that allows users to input their image and preferences for the room in text format.

[0361] The "means for inputting real environment data" refers to a means for inputting environmental data such as the actual dimensions and layout of the target room into the system.

[0362] "Means for generating / proposing layouts" refers to means for proposing layouts generated by the generation AI means to users.

[0363] The "means for outputting furniture / home appliance product information" is a means for providing the user with detailed information about the furniture and home appliances that will be arranged based on the proposed layout.

[0364] "Emotion engine means" is a technology for recognizing the current emotional state of a user.

[0365] The "means for adjusting layout proposals based on emotional state" refers to means for optimizing the generated layout proposals in response to the perceived emotional state of the user.

[0366] This invention is a system in which a user inputs a room image through a chat-based interface, and a generative AI model makes layout suggestions based on that input. Furthermore, by combining it with an emotion engine, it provides optimal layout suggestions according to the user's emotional state.

[0367] The server uses the following hardware and software to realize this system: For the generative AI model, it uses OpenAI's GPT-4, for example. For the emotion engine, it uses Affectiva's emotion recognition API, for example. By combining these technologies, it generates layout proposals based on the user's input and emotional state.

[0368] Using a chat-based interface (e.g., a web application or mobile app), a user inputs an image of the room they want, such as "a simple, calm living room." This input is sent to a generative AI model running on a server.

[0369] The generative AI model analyzes user input and combines it with previously learned room layout data to generate initial layout suggestions. For example, for a "simple and calm living room," it suggests wall colors, furniture placement, lighting types, etc.

[0370] The server then uses an emotion engine to recognize the user's current emotional state: if the user provides emotion data through the camera or microphone, the emotion engine analyzes that data to determine whether the user is relaxed or tense.

[0371] The server adjusts the generative AI model's initial layout proposals based on the emotional state obtained from the emotion engine. For example, if the user is relaxed, it will suggest furniture with more muted colors, and if the user is tense, it will suggest furniture arrangements that have a relaxing effect.

[0372] As a concrete example, consider a case where a user inputs that they want a "simple and relaxing living room." If the emotion engine recognizes that the user is relaxed, the generative AI model will generate the following layout suggestions:

[0373] The wall color is light blue

[0374] The sofa is gray

[0375] The lighting is warm-colored indirect lighting.

[0376] On the other hand, if the emotion engine detects that the user is nervous, the generative AI model will generate layout suggestions like this:

[0377] The wall color is pale green

[0378] The sofa is beige

[0379] The lighting is soft white light

[0380] Example prompt sentence:

[0381] User input: "A simple, calm living room."

[0382] Emotional state: Relaxed

[0383] Output of the generative AI model: "The walls are pale blue, the sofa is gray, and the lighting is warm, indirect lighting."

[0384] In this way, the server can generate and provide optimal room layout suggestions to the user based on the user's input and emotional state.

[0385] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0386] Step 1:

[0387] The user inputs the image of the room through a chat-based interface. The user then enters specific details in text format, such as "a living room with a simple and calm atmosphere." This input becomes the initial data for the system.

[0388] Input: Image of the user's room (e.g., "A simple, calm living room")

[0389] Output: Image data of the user's desired room

[0390] Step 2:

[0391] The server receives the user's input and sends it to the generative AI model. The server converts the user's input data into an appropriate format so that the generative AI model can analyze it.

[0392] Input: Image data of the room desired by the user

[0393] Output: Parsable data sent to a generative AI model

[0394] Step 3:

[0395] A generative AI model analyzes user input and generates initial layout suggestions. The model combines learned room layout data with user input to suggest wall colors, furniture placement, lighting types, and more.

[0396] Input: Parsable user image data

[0397] Output: Initial layout suggestions (e.g. wall color, furniture placement, lighting type)

[0398] Step 4:

[0399] The server uses an emotion engine to recognize the user's current emotional state. When the user provides emotion data through a camera or microphone, the emotion engine analyzes the data to determine whether the user is relaxed or tense.

[0400] Input: User emotion data (e.g. camera footage, audio data)

[0401] Output: User's emotional state (e.g. relaxed, tense)

[0402] Step 5:

[0403] The server adjusts the layout proposal of the generative AI model based on the emotional state. The server optimizes the initial layout proposal of the generative AI model based on the emotional state obtained from the emotion engine. For example, if the user is relaxed, it will suggest furniture with more subdued colors, and if the user is tense, it will suggest furniture arrangements that have a relaxing effect.

[0404] Input: initial layout proposal, user emotional state

[0405] Output: Final, adjusted layout proposal

[0406] Step 6:

[0407] The server sends the final layout proposal back to the user, who can review it through a chat-based interface and provide feedback if desired.

[0408] Input: Final tweaked layout proposal

[0409] Output: Present layout proposal to user

[0410] In this way, the server can generate and provide optimal room layout suggestions to the user based on the user's input and emotional state.

[0411] (Application example 1)

[0412] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0413] Conventional interior design proposal systems do not consider the user's emotional state when proposing layouts, making it difficult to increase the user's psychological satisfaction. Furthermore, in physical stores, there is a demand for optimal interior designs based on the user's emotions, but there has been a lack of effective means to achieve this.

[0414] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from the user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout that suits the real-world environment, a means for outputting product information of the furniture / appliances to be placed, an emotion engine means for recognizing the user's emotional state, and a means for adjusting the layout proposal based on the user's emotional state. This makes it possible to propose an optimal interior design that takes the user's emotional state into consideration.

[0415] "Generative AI means" is an artificial intelligence technology that learns various room layouts and generates new layout suggestions based on user input.

[0416] The "chat-based image input means" is an interface that allows users to input their desired image of the room in a chat format.

[0417] The "means for inputting real environment data" refers to a means for inputting the actual environment data of the target room (e.g., room dimensions, furniture arrangement, etc.).

[0418] The "means for generating / proposing layouts" is a means for generating a room layout suitable for the actual environment based on input data and proposing it to the user.

[0419] The "means for outputting product information" is a means for providing the user with information about the furniture and home appliances to be placed in the generated layout.

[0420] "Emotion engine means" is a technology for recognizing the user's emotional state and adjusting the system's behavior based on that information.

[0421] The "means for adjusting layout proposals based on emotional state" refers to a means for optimizing layout proposals by taking into account the emotional state of the user.

[0422] The following system configuration will be described as an embodiment of the present invention.

[0423] First, the server is equipped with a generation AI means that has learned various room layouts. This generation AI means analyzes the image of the desired room input by the user through a chat-based image input means, and combines it with the learned data to generate new layout proposals.

[0424] Users use their smartphones to chat with the system to input their desired image of the room. For example, they can input specific requests, such as "a relaxing cafe-style interior." This input is sent to the server and analyzed by the AI ​​generator.

[0425] Next, the server is provided with a means for inputting real-world environment data of the target room, such as the dimensions of the room and the existing furniture arrangement, provided by the user, which is used by the generative AI means to suggest an optimal layout.

[0426] Furthermore, the server is equipped with a means for generating and proposing layouts suited to the actual environment. Based on the user's preferences and actual environment data analyzed by the generation AI means, the server generates and proposes an optimal layout to the user. This proposal is displayed to the user via a smartphone application.

[0427] The server also has a means for outputting product information for the furniture and home appliances to be placed. Based on the generated layout, detailed information on the proposed furniture and home appliances is provided to the user, allowing the user to easily purchase the products required for the proposed layout.

[0428] Furthermore, the server is equipped with an emotion engine means for recognizing the user's emotional state. This emotion engine means analyzes the user's facial expressions and tone of voice to recognize the user's current emotional state. For example, it can determine whether the user is tense or relaxed.

[0429] Finally, the server includes a means for adjusting the layout proposal based on the user's emotional state. The generative AI means adjusts the layout proposed based on the user's emotional state recognized by the emotion engine means. For example, if the user is feeling tense, the server suggests colors and furniture arrangements that have a relaxing effect.

[0430] For example, if a user requests a "relaxing cafe-style interior" and their emotional state is recognized as "tense," the AI ​​generator will suggest colors and furniture arrangements that have a relaxing effect. An example of the prompt sentence in this case is as follows:

[0431] Desired interior design: A relaxing cafe-style interior

[0432] User's emotional state: nervous

[0433] Propose the best interior design for you.

[0434] In this way, it is possible to propose optimal interior designs based on the user's wishes and emotional state.

[0435] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0436] Step 1:

[0437] The user uses a smartphone application to input the image of the room they want in chat format. For example, they might input "a relaxing cafe-style interior." This input data is then sent to the server.

[0438] Step 2:

[0439] The server receives image input data sent by the user, analyzes the received data, and generates a prompt sentence to input into the generative AI model. Specifically, it creates a prompt sentence containing the user's desired interior design.

[0440] Step 3:

[0441] The server uses the generative AI model to generate layout proposals based on the prompt sentence. The generative AI model generates a new layout that meets the user's wishes based on the learned room layout data. This generated layout proposal is saved on the server.

[0442] Step 4:

[0443] The user inputs the real-world environment data of the target room (e.g., room dimensions, existing furniture layout, etc.) through a smartphone application, and this data is also sent to the server.

[0444] Step 5:

[0445] The server receives the real-world data sent by the user, analyzes the received data, and combines it with the generated layout proposal to generate an optimal layout suited to the real-world environment. This optimized layout proposal is then stored in the server.

[0446] Step 6:

[0447] The server uses an emotion engine means to recognize the user's emotional state. It analyzes the user's facial expressions and tone of voice to recognize the user's current emotional state. For example, it determines whether the user is tense or relaxed.

[0448] Step 7:

[0449] The server adjusts the generated layout proposal based on the user's emotional state recognized by the emotion engine means. For example, if the user is nervous, the server may suggest colors and furniture arrangements that have a relaxing effect. The adjusted layout proposal is stored in the server.

[0450] Step 8:

[0451] The server generates product information for the furniture and home appliances to be arranged based on the adjusted layout proposal, and transmits the generated product information to the smartphone application to be provided to the user.

[0452] Step 9:

[0453] The user receives the adjusted layout proposal and product information sent from the server through a smartphone application, allowing the user to check and purchase the products required for the proposed layout.

[0454] In this way, an optimal interior design proposal based on the user's wishes and emotional state is realized.

[0455] Example 2

[0456] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0457] Conventional room layout suggestion systems make uniform suggestions without considering the user's emotional state, which fails to sufficiently increase user satisfaction. Furthermore, they lack the functionality to analyze real-world data to suggest optimal layouts, making it difficult to provide specific suggestions tailored to the user's needs. Furthermore, product selection suggestions do not reflect the user's emotional state, failing to stimulate the user's purchasing motivation.

[0458] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0459] In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from the user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout that suits the real environment, a means for outputting product information of the furniture / appliances to be placed, an emotion engine means for recognizing the emotional state of the user, and a means for suggesting product selection based on the emotional state. This makes it possible to propose optimal layouts and product selections that take the user's emotional state into consideration.

[0460] "Generative AI means" is an artificial intelligence technology that learns various room layouts and generates the optimal layout for the user.

[0461] A "chat-based image input means" is an interface that allows users to input images and requests in text format.

[0462] "Means for inputting real-world data" refers to the means for inputting real-world data such as LiDAR data, images, and videos of the target room into the system.

[0463] "Means for generating / proposing layouts suited to the actual environment" refers to means for generating and proposing layouts that are optimal for users based on inputted actual environment data.

[0464] The "means for outputting product information of the furniture / home appliances to be arranged" is a means for providing the user with detailed information about the furniture and home appliances to be arranged based on the proposed layout.

[0465] "Emotion engine means" is a technology for recognizing the current emotional state of a user.

[0466] The "means for suggesting product selection based on emotional state" is a means for suggesting optimal products according to the recognized emotional state of the user.

[0467] This invention is a system for users to optimize their room layout, and it combines a generative AI model and an emotion engine to make product recommendations based on the user's emotional state. Specific embodiments of this system are described below.

[0468] 1. Data Entry

[0469] Users access the system interface using a smartphone or PC, select real-world data such as LiDAR data, images, and videos, and click the upload button. This allows users to input the real-world data of the target room into the system.

[0470] 2. Data transmission

[0471] The terminal sends the data selected by the user to the server using a secure protocol (e.g., HTTPS). When the transmission is complete, the terminal notifies the user that "Data transmission is complete."

[0472] 3. Data Analysis

[0473] The server analyzes the received data. The LiDAR data is used to reconstruct the dimensions and shape of the room as a 3D model using OpenCV. Images and videos are used to recognize colors and furniture placement using TENSORFLOW (registered trademark). The analysis results are stored in an internal database.

[0474] 4. Generating layout proposals

[0475] The server generates layout proposals using a generative AI model (e.g., GPT-4 or DALL-E) based on the analysis results. The proposals are sent to the user's device and displayed on the interface. The user can review the proposed layout and make any necessary modifications.

[0476] 5. Emotion Recognition by Emotion Engine

[0477] When a user types "I'm looking for a new TV," the device sends this information to a server, which uses the Emotion API to analyze the user's facial images and voice data to recognize their current emotional state.

[0478] 6. Emotion-based product recommendations

[0479] The server then suggests product choices based on the results of the emotion engine. For example, if the user is feeling happy, it might suggest the latest large-screen 4K TV, and if they are feeling sad, it might suggest a cost-effective 32-inch TV. The suggestions are sent to the user's device and displayed on the interface.

[0480] Examples of concrete examples and prompts

[0481] As a concrete example, consider a case where a user wants to change the layout of their room. The user inputs LiDAR data and images of the room into the system. The server analyzes this data and uses a generative AI model to suggest new furniture placements. Furthermore, if the user inputs "I'm looking for a new TV," the emotion engine recognizes the user's emotional state and suggests an appropriate TV.

[0482] Example prompt sentence:

[0483] "I would like to change the layout of my room, so please analyze the following data and suggest a new layout. I am attaching LiDAR data and room images. Also, I am looking for a new TV, so please suggest a TV based on my emotional state."

[0484] In this way, the system works in cooperation with the user, terminal, and server, and can propose optimal layouts and products to the user.

[0485] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0486] Step 1: Data entry

[0487] Users access the system's interface using a smartphone or PC. On the interface, they select real-world data, such as LiDAR data, images, or videos, and click the upload button. This allows the user to input the real-world data of the target room into the system. The input data includes the room's dimensions, shape, color, furniture arrangement, etc.

[0488] Step 2: Sending data

[0489] The terminal sends the data selected by the user to the server using a secure protocol (e.g., HTTPS). Once the transmission is complete, the terminal notifies the user that "Data transmission is complete." The input data is then sent to the server and prepared for analysis.

[0490] Step 3: Analyze the data

[0491] The server analyzes the received data. The LiDAR data is used to reconstruct the dimensions and shape of the room as a 3D model using OpenCV. Images and videos are used to recognize colors and furniture placement using TensorFlow. The analysis results are stored in an internal database and used in the next step. The input data is output as a 3D model and recognition results.

[0492] Step 4: Generate layout proposals

[0493] The server generates layout proposals using a generative AI model (e.g., GPT-4 or DALL-E) based on the analysis results. The generated proposals are sent to the user's device and displayed on the interface. The user can review the proposed layout and make corrections as necessary. The input data is the analysis results, and the output data is the layout proposals.

[0494] Step 5: Emotion Recognition with the Emotion Engine

[0495] When a user types "I'm looking for a new TV," the device sends this information to the server. The server uses the Emotion API to analyze the user's facial image and voice data to recognize their current emotional state. The input data is the user's text input and facial image and voice data, and the output data is their emotional state.

[0496] Step 6: Emotion-based product recommendations

[0497] The server then suggests product selections based on the results of the emotion engine. For example, if the user is feeling happy, it suggests the latest large-screen 4K TV, and if they are feeling sad, it suggests a cost-effective 32-inch TV. The suggestions are sent to the user's device and displayed on the interface. The input data is the emotional state, and the output data is the product suggestions.

[0498] (Application example 2)

[0499] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0500] Conventional layout suggestion systems simply make layout suggestions based on real-world data without taking into account the user's emotional state. As a result, they are unable to make optimal product or layout suggestions based on the user's emotional or psychological state, and this does not sufficiently improve the user experience. Furthermore, there has been no system that can suggest optimal product placement or layouts in a physical store. This makes it difficult to maximize users' purchasing motivation.

[0501] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0502] In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout that suits the real environment, a means for outputting product information for furniture / appliances to be placed, an emotion engine means for analyzing the emotional state of the user, a means for proposing products based on the emotional state, and a means for proposing optimal product placement and layout in a physical store. This makes it possible to propose optimal products and layouts according to the user's emotional state, thereby improving the shopping experience in a physical store.

[0503] "Generative AI means" is an artificial intelligence technology that learns various room layouts and generates layouts based on the real environment.

[0504] A "chat-based image input means" is an interface that allows users to input images and requests in text format.

[0505] "Means for inputting real-world data" refers to the means for incorporating actual data (e.g., LiDAR data, images, videos) of the target room or space into the system.

[0506] The "means for generating / proposing layouts" is a means for generating an optimal layout based on input real-world data and proposing it to the user.

[0507] The "means for outputting product information" is a means for providing the user with information about the furniture and home appliances to be placed in the generated layout.

[0508] The "emotion engine means" is a technology for analyzing the user's emotional state and making appropriate suggestions based on that state.

[0509] "Means for suggesting products" refers to means for selecting and suggesting optimal products based on the user's emotional state.

[0510] "Means for proposing optimal product placement and layout within a physical store" refers to a means for analyzing environmental data within a physical store and proposing optimal product placement and layout.

[0511] A system for carrying out this invention has the following configuration: A server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout that suits the real environment, a means for outputting product information for furniture / home appliances to be placed, an emotion engine means for analyzing the emotional state of the user, a means for proposing products based on the emotional state, and a means for proposing optimal product placement and layout in a physical store.

[0512] Hardware and Software Configuration

[0513] Hardware:

[0514] Smartphone camera

[0515] A LiDAR sensor connected to a smartphone (or a LiDAR built into the smartphone)

[0516] software:

[0517] OpenCV (camera data acquisition and processing)

[0518] EmotionRecognizer (a custom library for emotion recognition)

[0519] LayoutGenerator (a custom library for generating layouts)

[0520] ProductRecommender (a custom library for product recommendations)

[0521] System Operation

[0522] As users walk around the store with their smartphones, they use the camera and LiDAR sensor to collect real-world data. The smartphone camera captures video in real time and processes the image data using OpenCV. The LiDAR sensor then captures 3D data of the target room or space.

[0523] The server analyzes the captured camera data using EmotionRecognizer to recognize the user's emotional state. For example, if the user is smiling, it is recognized as "joy," and if they are frowning, it is recognized as "sadness."

[0524] Next, the server analyzes the acquired LiDAR data using LayoutGenerator and generates a layout that is optimal for the real environment. The generated layout is displayed on the user's smartphone, allowing the user to check the proposal.

[0525] Furthermore, the server uses an emotion engine means to suggest products based on the user's emotional state. For example, if the user is feeling "joy," the server suggests high-quality products, and if the user is feeling "sad," the server suggests cost-effective products.

[0526] Specific examples

[0527] As a concrete example, consider a scenario in which a user walks around a store with their smartphone, and an application makes layout and product suggestions in real time. In this case, the user collects real-world data using the smartphone's camera and LiDAR sensor, and the server analyzes that data to make optimal layout and product suggestions.

[0528] Example prompt sentence:

[0529] "As users walk around the store with their smartphones, they use cameras and LiDAR sensors to collect data on the real-world environment. The application analyzes this data and suggests optimal product placement and layout within the store. It also analyzes the user's emotional state in real time and makes product recommendations based on that emotion."

[0530] In this way, it becomes possible to propose optimal products and layouts according to the user's emotional state, improving the shopping experience in physical stores.

[0531] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0532] Step 1:

[0533] Users walk around the store with their smartphones, and real-world data is collected using cameras and LiDAR sensors.

[0534] Input: Smartphone camera footage and LiDAR sensor data.

[0535] Output: Camera image data and LiDAR 3D data.

[0536] How it works: The smartphone camera captures video in real time, and the LiDAR sensor acquires three-dimensional data of the target room or space.

[0537] Step 2:

[0538] The server receives the camera data and processes the image data using OpenCV.

[0539] Input: Camera video data.

[0540] Output: Processed image data.

[0541] Specific operation: The server receives the camera image data sent from the smartphone and performs image preprocessing (e.g., noise removal, resolution adjustment) using the OpenCV library.

[0542] Step 3:

[0543] The server uses EmotionRecognizer to analyze the user's emotional state from the processed image data.

[0544] Input: Processed image data.

[0545] Output: The user's emotional state (e.g., happy, sad).

[0546] Specific operation: The server uses the EmotionRecognizer library to extract the user's facial features from the processed image data and recognize their emotional state.

[0547] Step 4:

[0548] The server receives the LiDAR data and uses LayoutGenerator to generate a layout that is optimal for the real environment.

[0549] Input: 3D LiDAR data.

[0550] Output: The generated layout proposal.

[0551] Specific operation: The server receives LiDAR data sent from the smartphone and uses the LayoutGenerator library to generate an optimal layout based on the real environment.

[0552] Step 5:

[0553] The server sends the generated layout to the user's smartphone, where the user can check it.

[0554] Input: The generated layout proposal.

[0555] Output: Layout proposals displayed on the user's smartphone.

[0556] Specific operation: The server sends the generated layout proposal to the user's smartphone, and the user checks the proposed layout on the smartphone screen.

[0557] Step 6:

[0558] The server uses an emotion engine means to make product suggestions based on the user's emotional state.

[0559] Input: The user's emotional state.

[0560] Output: Product suggestions based on emotional state.

[0561] Specific operation: The server uses the emotion engine means to select and suggest the most suitable products based on the user's emotional state.

[0562] Step 7:

[0563] The server sends product suggestions to the user's smartphone, which the user confirms.

[0564] Input: Product suggestions based on emotional state.

[0565] Output: Product suggestions displayed on the user's smartphone.

[0566] Specific operation: The server sends product suggestions based on the user's emotional state to the user's smartphone, and the user checks the suggested products on the smartphone screen.

[0567] Example 3

[0568] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0569] Conventional layout suggestion systems have difficulty making optimal suggestions based on user emotions and preferences. Furthermore, they have been unable to make optimal suggestions based on behavioral economics for product and shelf layout in commercial facilities. This has led to the issue of not being able to fully stimulate user satisfaction and purchasing motivation.

[0570] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0571] In this invention, the server includes a generation AI means, a chat-based image input means from the user, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout suited to the real environment, a means for outputting product information of the furniture / appliances to be arranged, an emotion engine means for analyzing the user's emotions, and a means for optimizing the layout proposal based on the analysis results of the emotion engine. This makes it possible to propose an optimal layout based on the user's emotions and preferences, and also to propose product and shelf placements based on behavioral economics.

[0572] "Generative AI methods" refer to artificial intelligence techniques that generate layout suggestions based on user input.

[0573] "Chat-based image input means from users" refers to an interface that allows users to input layout preferences and images in chat format.

[0574] The "means for inputting actual environment data of the target room" refers to a means for inputting actual environment data such as the dimensions and layout of the target room.

[0575] "Means for generating / proposing layouts suited to the actual environment" refers to means for generating and proposing optimal layouts based on inputted actual environment data.

[0576] "Means for outputting product information of furniture / appliances to be arranged" refers to means for outputting specific product information (manufacturer name, product name, price, etc.) of furniture and appliances included in the proposed layout.

[0577] "Emotion engine means for analyzing user emotions" refers to technology for analyzing user feedback and reactions and identifying emotions.

[0578] "Means for optimizing layout proposals based on the analysis results of the emotion engine" refers to means for generating optimal layout proposals based on the analysis results of the emotion engine in accordance with the user's emotions and preferences.

[0579] "Extension function that suggests product / shelf layout methods based on behavioral economics" refers to an extension function that suggests optimal product and shelf layout methods in commercial facilities, etc., based on the theory of behavioral economics.

[0580] This invention is a system that provides users with layout suggestions, including specific product information for furniture and home appliances. It also includes an extension function that makes optimal suggestions based on behavioral economics for product and shelf placement in commercial facilities. The system uses an emotion engine to learn changes in user emotions, and can optimize layout suggestions and product selection suggestions based on the results.

[0581] Hardware and software used

[0582] Hardware: Servers, user devices (PCs, smartphones, tablets)

[0583] Software: Generative AI models (e.g., GPT-4), emotion engines, database management systems (e.g., MySQL®), web application frameworks (e.g., Django)

[0584] Specific operation of the system

[0585] 1. Accepting user input

[0586] A user uses a device to input a request for a layout suggestion, for example, "Please suggest a layout for my living room."

[0587] 2. Generating layout proposals using a generative AI model

[0588] The server uses a generative AI model (e.g., GPT-4) to generate layout proposals based on the user's requests, including specific furniture and appliance information (e.g., manufacturer name, product name, price, etc.).

[0589] 3. The server generates layout suggestions and sends them to the user.

[0590] The server sends the layout proposals received from the generative AI model to the user's device, where the user can view the proposals.

[0591] 4. User reviews the proposal and provides emotional feedback

[0592] The user reviews the proposed layout and provides emotional feedback, for example, "I like this sofa." The user's feedback is sent to the server.

[0593] 5. User sentiment analysis using an emotion engine

[0594] The server analyzes the user feedback using an emotion engine, which identifies the user's emotions (happiness, interest, indifference, etc.) and stores the results in a database.

[0595] 6. Emotion-based recommendation optimization

[0596] The server sends a new prompt to the generative AI model based on the analysis results of the emotion engine. The prompt contains optimization instructions based on the user's emotions. The generative AI model then generates an optimized layout proposal.

[0597] 7. The server sends the optimized offer to the user

[0598] The server sends the optimized layout proposal received from the generative AI model to the user's device, where the user can view the optimized proposal.

[0599] Examples of concrete examples and prompts

[0600] Examples:

[0601] User types, "Suggest a living room layout."

[0602] The server uses the generated AI model to generate layout proposals that include specific product information such as "sofa (manufacturer name: general manufacturer, product name: general product, price: 50,000 yen) and television (manufacturer name: general manufacturer, product name: general product, price: 80,000 yen)."

[0603] A user types, "I love this sofa," and the emotion engine learns the emotion of joy.

[0604] Based on the learning results of the emotion engine, the server will prioritize suggestions for sofas and related products in the same category.

[0605] Example prompt sentence:

[0606] "Please suggest a living room layout. Please include specific information about the furniture and appliances (manufacturer, product name, price, etc.)."

[0607] "I love this sofa. Are there any other sofas you'd recommend?"

[0608] In this way, the user can receive an optimal layout proposal based on their emotions while referring to specific product information. The flow of the specification process in the third embodiment will be described with reference to FIG.

[0609] Step 1:

[0610] A user uses a terminal to input a request for a layout proposal. For example, the user may input "Please suggest a layout for the living room." The input request is sent to the server. The server receives the request and proceeds to the next processing step.

[0611] Step 2:

[0612] The server sends a prompt to a generative AI model (e.g., GPT-4). The prompt contains the user's request. The generative AI model generates a layout proposal based on the prompt. The generated layout proposal includes specific furniture and appliance information (manufacturer name, product name, price, etc.). The server receives the output from the generative AI model and proceeds to the next processing step.

[0613] Step 3:

[0614] The server sends the layout proposal received from the generative AI model to the user's device, where the user can review the proposal. After the user has reviewed the proposal, the process proceeds to the next step.

[0615] Step 4:

[0616] The user reviews the proposed layout and provides emotional feedback, for example, "I like this sofa." The feedback is sent to the server, which receives it and proceeds to the next processing step.

[0617] Step 5:

[0618] The server uses the emotion engine to analyze the user's feedback. The emotion engine identifies the user's emotion (happiness, interest, indifference, etc.) and stores the result in a database. The server receives the analysis result from the emotion engine and proceeds to the next processing step.

[0619] Step 6:

[0620] The server sends a new prompt to the generative AI model based on the analysis results of the emotion engine. The prompt contains optimization instructions based on the user's emotions. The generative AI model generates an optimized layout proposal based on the new prompt. The server receives the output from the generative AI model and proceeds to the next processing step.

[0621] Step 7:

[0622] The server sends the optimized layout proposal received from the generative AI model to the user's device, where the user can check the optimized proposal on their device. This allows the user to receive the optimal layout proposal based on their emotions while referring to specific product information.

[0623] (Application example 3)

[0624] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0625] Conventional layout suggestion systems did not take into account user emotions or purchasing history when making suggestions, making it difficult to increase user satisfaction. Furthermore, they lacked the functionality to perform real-time emotion analysis and recommend products based on that analysis, making it difficult to maximize users' purchasing motivation. Furthermore, they lacked product and shelf layout suggestions based on behavioral economics, making it difficult to achieve optimal product placement in commercial facilities.

[0626] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0627] In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from users, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout that suits the real-world environment, a means for outputting product information for the furniture / appliances to be placed, an emotion recognition means for analyzing the user's emotions in real time, a product recommendation means for recommending products based on emotions, and a learning means for learning the user's purchasing history and emotional data and optimizing future proposals. This enables optimal layout proposals and product recommendations that take into account the user's emotions and purchasing history, thereby increasing user satisfaction. Furthermore, by proposing product and shelf placements based on behavioral economics, optimal product placement in commercial facilities can be achieved.

[0628] A "generative AI means" is an artificial intelligence that learns various room layouts and generates the optimal layout based on the user's requirements.

[0629] "Chat-based image input means" refers to an interface that allows users to input images and requests in text format.

[0630] "Means for inputting real-world data" refers to the means for acquiring actual data of the target room or space and inputting it into the system.

[0631] "Means for generating / proposing layouts" refers to means for generating optimal layouts based on input data and proposing them to users.

[0632] The "means for outputting product information" refers to a means for providing the user with specific product information about the furniture and home appliances to be placed.

[0633] An "emotion recognition means" is a means for analyzing a user's emotions in real time and reflecting the results in the system.

[0634] A "product recommendation means" is a means for recommending optimal products based on the user's emotions.

[0635] "Learning methods" are means for learning from a user's purchasing history and emotional data and optimizing the next suggestion.

[0636] A system for carrying out this invention has the following configuration: The server includes a generation AI means that has learned various room layouts, a chat-based image input means from the user, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout that suits the real-world environment, a means for outputting product information about the furniture / home appliances to be placed, an emotion recognition means that analyzes user emotions in real time, a product recommendation means that recommends products based on emotions, and a learning means that learns the user's purchasing history and emotional data and optimizes the next proposal.

[0637] Hardware and software used

[0638] Hardware: Smartphone camera, server

[0639] Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition model), ProductRecommender (product recommendation system)

[0640] Data processing and calculation

[0641] 1. User Input: The user inputs room images and requirements using a chat-based interface on their smartphone, and also captures real-world room data using the smartphone camera.

[0642] 2. Generative AI: The server uses a generative AI model to generate the optimal layout based on the input data. This generative AI model has learned various room layouts.

[0643] 3. Emotion recognition: Using the user's facial image captured by the smartphone camera, EmotionRecognizer analyzes the user's emotions in real time.

[0644] 4. Product Recommendation: Based on the emotional data obtained by the emotion recognition method, ProductRecommender recommends the most suitable product. This recommendation is based on the results of learning the user's purchasing history and emotional data.

[0645] 5. Output method: The generated layout and information on recommended products (manufacturer name, product name, price, etc.) are displayed on the user's smartphone.

[0646] Specific examples

[0647] When a user uses their smartphone in a store to scan the shelves with their camera, their emotions are analyzed in real time and products that they might be interested in are recommended. For example, if the user smiles, related products (e.g., snacks or drinks) are recommended based on that emotion.

[0648] Prompt Sentence Examples

[0649] "Develop an application that analyzes emotions in real time when a user scans a shelf with their smartphone camera and recommends products based on their emotions. Use the EmotionRecognizer model for emotion recognition and the ProductRecommender system for product recommendations."

[0650] In this way, it becomes possible to propose optimal layouts and recommend products that take into account the user's emotions and purchasing history, thereby increasing user satisfaction.In addition, by proposing product and shelf placement based on behavioral economics, it is possible to achieve optimal product placement in commercial facilities.

[0651] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0652] Step 1:

[0653] A user uses a chat-based interface on their smartphone to input their image of the room and their requirements, and the text data is sent to a server, which receives the data and passes it to a generative AI model.

[0654] Step 2:

[0655] The user captures real-world data of the room using the smartphone camera. The captured image data is sent to the server, which receives the data and stores it as real-world data.

[0656] Step 3:

[0657] The server uses a generative AI model to generate an optimal layout based on the input text data and real-world data, and the generated layout data is stored within the server.

[0658] Step 4:

[0659] The user captures a facial image using the smartphone camera. The captured facial image is then sent to the server, which receives the data and passes it to the emotion recognition unit.

[0660] Step 5:

[0661] The server uses the emotion recognition means to analyze the user's emotions from the captured facial image in real time, and the emotion data obtained as a result of the analysis is stored in the server.

[0662] Step 6:

[0663] The server uses the product recommendation means to recommend optimal products based on the emotion data obtained by the emotion recognition means, and the recommended product data is stored in the server.

[0664] Step 7:

[0665] The server then sends the generated layout data and recommended product data to the user's smartphone, which receives and displays the data on its screen.

[0666] Step 8:

[0667] When a user selects a recommended product, the selection data is sent to the server, which receives the data and stores it as the user's purchasing history.

[0668] Step 9:

[0669] The server learns the user's purchasing history and emotional data and uses learning methods to optimize the next recommendation. The learning results are stored in the server and reflected in the next recommendation.

[0670] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0671] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0672] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0673] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0674] [Second embodiment]

[0675] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0676] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0677] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0678] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0679] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0680] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0681] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0682] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0683] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program.

[0684] The processor 28 executes the specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0685] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0686] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0687] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0688] "Example 1"

[0689] In one example of the present invention, a user inputs an image of their room (e.g., "a living room with a simple and calm atmosphere") through a chat-based interface. This input is analyzed by the generative AI and combined with previously learned room layout data to generate new layout proposals.

[0690] "Example 2"

[0691] In addition, users input real-world environmental data (e.g., data captured by LiDAR, images, videos, etc.) into the system. This data is analyzed by generative AI, and a layout proposal tailored to the real environment is generated.

[0692] "Example 3"

[0693] The generated layout proposal includes specific product information (e.g., manufacturer name, product name, price, etc.) for the furniture and home appliances to be placed, allowing users to use this information as a reference for specific product selection.

[0694] "Example 4"

[0695] Furthermore, an example embodiment of the present invention includes an extension function that makes optimal suggestions based on behavioral economics regarding the placement of products and product shelves in commercial facilities, etc. This function makes it possible to propose optimal product placement that encourages customer purchasing behavior.

[0696] The processing flow of each embodiment will be described below.

[0697] "Example 1"

[0698] Step 1: The user inputs an image of their room (e.g., "a living room with a simple and calm atmosphere") through a chat-based interface.

[0699] Step 2: This input is parsed by the generative AI.

[0700] Step 3: The generation AI combines the learned room layout data with input from the user to generate new layout proposals.

[0701] "Example 2"

[0702] Step 1: The user inputs real-world environmental data (e.g., data captured by LiDAR, images, videos, etc.) of the target room into the system.

[0703] Step 2: This data is analyzed by the generative AI.

[0704] Step 3: The generation AI combines the real-world data with the learned room layout data to generate layout proposals tailored to the real-world environment.

[0705] "Example 3"

[0706] Step 1: The generated layout proposal includes specific product information for the furniture and appliances to be placed.

[0707] (Example: Manufacturer name, product name, price, etc.)

[0708] Step 2: The user uses this product information to make a specific product selection.

[0709] "Example 4"

[0710] Step 1: The system will include an extension that makes optimal suggestions based on behavioral economics regarding product and shelf placement in commercial facilities, etc.

[0711] Step 2: This feature suggests optimal product placement to encourage customer purchasing behavior.

[0712] Example 1

[0713] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0714] Conventional room layout suggestion systems have difficulty proposing layouts that reflect the user's specific image, and making suggestions that are appropriate for the actual environment. Furthermore, they lack specific furniture and appliance suggestions based on the user's desired atmosphere and style.

[0715] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0716] In this invention, the server includes a chat-based image input means for a user, a means for transmitting the input image to the server, a means for the server to pass the received image to a generative AI model, a means for the generative AI model to analyze the input and combine it with learned layout data to generate a new layout proposal, a means for the server to transmit the generated layout proposal to a terminal, and a means for the terminal to display the layout proposal to the user. This makes it possible to propose a layout based on the user's specific image, and also to propose specific furniture and home appliances suitable for the actual environment.

[0717] The "chat-based image input means from the user" is a means for providing an interface for the user to input an image of the room in text format.

[0718] The "means for transmitting the input image to the server" is a communication means for transmitting the text data input by the user to the server.

[0719] The "means for passing the image received by the server to the generative AI model" is a processing means for passing the text data received by the server to the generative AI model.

[0720] "The means by which the generative AI model analyzes input and combines it with previously learned layout data to generate new layout proposals" refers to the means by which the generative AI model analyzes user input and combines it with previously learned data to generate new layout proposals.

[0721] The "means by which the server transmits the generated layout proposal to the terminal" refers to a communication means by which the server transmits the layout proposal received from the generative AI model to the terminal.

[0722] The "means for the terminal to display the layout proposal to the user" refers to a display means for the terminal to display the layout proposal received from the server to the user.

[0723] This invention is a system in which a user inputs an image of a room through a chat-based interface, and a generative AI model generates new layout proposals based on that image. Specific embodiments of this system are described below.

[0724] Users access the chat interface using a smartphone or PC browser, which allows them to input a text image of the room they want to live in. For example, they might enter "a living room with a simple and relaxing atmosphere."

[0725] The device sends the text data entered by the user to the server as an HTTP POST request. This request includes the user's input text. The server receives the HTTP POST request, extracts the user's input text from the request body, and then creates an API request to pass this text to the generative AI model and sends it to the generative AI model.

[0726] The generative AI model uses a model that uses natural language processing technology (e.g., GPT-4). This model analyzes the user's input text and combines it with previously learned room layout data to generate new layout proposals. Based on the analysis results, it can generate layout proposals such as the following:

[0727] The wall color is light gray

[0728] The sofa has a simple design and is beige in color.

[0729] The coffee table is made of wood and has a natural color.

[0730] The lighting makes extensive use of indirect lighting to provide soft light.

[0731] The server sends the layout proposal received from the generative AI model to the device as an HTTP response. This response contains the generated layout proposal. The device parses the HTTP response received from the server and extracts the layout proposal. The device then displays the proposal on the chat interface. The user can review the proposed layout and re-enter it if necessary.

[0732] For example, if a user inputs "I want to create a simple and calm living room," a generative AI model might generate the following layout suggestions:

[0733] The wall color is light gray

[0734] The sofa has a simple design and is beige.

[0735] The coffee table is made of wood and has a natural color.

[0736] The lighting makes extensive use of indirect lighting to provide soft light.

[0737] Example prompt sentence:

[0738] User: I want to create a simple and calm living room.

[0739] Server: Analyzing. Please wait a moment.

[0740] Generative AI model: The wall color is pale gray, the sofa is simple and beige, the coffee table is made of wood and has a natural color, and the lighting is indirect, providing soft light.

[0741] Server: The proposed layout is as follows:

[0742] Terminal: The wall color is pale gray, the sofa is simply beige, the coffee table is made of wood and has a natural color, and the lighting is indirect, providing soft light.

[0743] In this way, users can easily receive room layout suggestions through a chat-based interface.

[0744] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0745] Step 1:

[0746] The user inputs an image of the room into the chat interface.

[0747] The user opens a browser on their smartphone or PC, accesses the chat interface, types in, for example, "a living room with a simple and calm atmosphere," and clicks the send button. The input data is in text format.

[0748] Step 2:

[0749] The device sends input to the server.

[0750] The terminal sends the text data entered by the user to the server as an HTTP POST request. This request contains the user's input text. The input data is in text format, and the output data is an HTTP request.

[0751] Step 3:

[0752] The server receives the input and passes it to the generative AI model.

[0753] The server receives the HTTP POST request, extracts the user's input text from the request body, creates an API request to pass this text to the generative AI model, and sends it to the model. The input data is in text format, and the output data is an API request.

[0754] Step 4:

[0755] The generative AI model analyzes the input and combines it with learned layout data to generate new layout proposals.

[0756] The generative AI model receives API requests and analyzes the user's input text. Based on the analysis results, it combines them with the learned room layout data to generate new layout proposals. For example, it generates proposals such as "The wall color should be light gray, the sofa should have a simple beige design, the coffee table should be made of wood and have a natural color, and the lighting should be indirect, providing soft light." The input data is in text format, and the output data is the layout proposal.

[0757] Step 5:

[0758] The server receives the generated layout proposal and transmits it to the terminal.

[0759] The server sends the layout proposal received from the generative AI model to the terminal as an HTTP response. This response contains the generated layout proposal. The input data is the layout proposal, and the output data is the HTTP response.

[0760] Step 6:

[0761] The terminal displays the layout proposal to the user.

[0762] The device analyzes the HTTP response received from the server and extracts layout suggestions. It then displays these suggestions on the chat interface. The user can review the proposed layout and re-enter it if necessary. The input data is the HTTP response, and the output data is the displayed layout suggestions.

[0763] (Application example 1)

[0764] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0765] Conventional interior design support systems have difficulty generating layout proposals that specifically reflect the user's desired image of the space. Furthermore, there is a lack of tools to efficiently support the interior design of brick-and-mortar stores, and there has been a demand for a system that allows users to easily receive layout proposals using their smartphones.

[0766] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0767] In this invention, the server includes a generative AI means that has learned the layouts of various spaces, a chat-based image input means from a user, a means for inputting real-world environmental data of the target space, a means for generating / proposing a layout that suits the real environment, a means for outputting product information of items to be placed, a means for supporting the interior design of a physical store, a means including an application to be installed on a smartphone, and a means for generating a layout proposal using the generative AI model. This makes it possible to generate a layout proposal that specifically reflects the image of the space desired by the user, and to efficiently support the interior design of a physical store.

[0768] "Generative AI means" is an artificial intelligence technology that learns the layouts of various spaces and generates new layout proposals based on user input.

[0769] The "chat-based image input means" is an interface that allows users to input an image of the desired space in text format.

[0770] "Means for inputting real-world data" refers to a means for acquiring the current state of the target space as data and inputting it into the system.

[0771] "Means for generating / proposing layouts" refers to means for proposing new layouts to users based on data analyzed by the generation AI means.

[0772] The "means for outputting product information of an item" is a means for providing the user with detailed information about the items to be arranged in the layout.

[0773] "Means to support the interior design of physical stores" refers to means that provide support for efficiently designing the space of physical stores.

[0774] "Means including an application installed on a smartphone" means means that allows a user to use the system through an application that runs on a smartphone.

[0775] "Means for generating layout proposals using a generative AI model" means means for generating layout proposals based on user input using a generative AI model.

[0776] A system for implementing this invention is configured as follows: First, a user uses an application installed on a smartphone to input an image of a desired space into a chat-based interface. This input is analyzed by a generative AI means. The generative AI means has learned the layouts of various spaces and generates new layout proposals based on the user's input.

[0777] The server receives image input from the user and acquires current spatial data using a means of inputting real-world environmental data for the target space. This data can be collected, for example, using a smartphone camera or sensors. The server then uses generative AI means to generate and propose a layout that fits the real-world environment.

[0778] The generated layout proposal is displayed to the user through a smartphone application, and product information for the items to be placed is also displayed at the same time. This allows the user to purchase the necessary items based on the proposed layout.

[0779] As a concrete example, if the user requests a "cafe with a simple and relaxing atmosphere," the following prompt sentence is generated:

[0780] Example prompt sentence:

[0781] The image of the store desired by users: A simple and calm cafe

[0782] Generate a new layout proposal.

[0783] By inputting this prompt into a generative AI model, new store layout proposals that match the user's preferences can be obtained. For example, OpenAI's GPT-3 model can be used as a generative AI model. This model analyzes the user's input and generates optimal layout proposals based on the data it has already learned.

[0784] This system allows users to easily receive interior design suggestions using their smartphones, making it possible to efficiently design the interiors of physical stores.

[0785] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0786] Step 1:

[0787] Users launch an application installed on their smartphone and input an image of the space they want into a chat-based interface. The input data is in text format and includes a specific image, such as "a cafe with a simple and relaxing atmosphere." This input data is then passed on to the next step of processing.

[0788] Step 2:

[0789] The terminal receives image input from the user and sends it to the generation AI means. The generation AI means analyzes the input text data and generates a prompt sentence. This prompt sentence is converted into a format that can be input to the generation AI model. For example, the prompt sentence might be, "The image of the store desired by the user: A cafe with a simple and relaxing atmosphere. Please generate a new layout proposal."

[0790] Step 3:

[0791] The server inputs the generated prompt sentence into a generative AI model (e.g., OpenAI's GPT-3 model), which analyzes the prompt sentence and generates new layout proposals based on the learned data. These layout proposals are then passed to the next step.

[0792] Step 4:

[0793] The server receives the generated layout proposal and combines it with real-world data of the target space. The real-world data is acquired using the smartphone's camera and sensors. This data includes the space's dimensions and the layout of existing furniture. The server integrates this data and generates an optimal layout proposal.

[0794] Step 5:

[0795] The server sends the final layout proposal to the smartphone application. The device displays this proposal to the user. The user can review the proposed layout and make any necessary modifications. Product information for the items to be placed is also displayed at the same time, allowing the user to purchase the items they need.

[0796] Step 6:

[0797] If the user is satisfied with the proposed layout, they finalize it. The device then sends this information to the server, which then performs subsequent processes such as saving the layout and ordering items. This allows the user to efficiently realize their interior design.

[0798] Example 2

[0799] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0800] Conventional room layout proposal systems are limited in the types and formats of data that users can input, making it difficult to provide specific proposals that are suited to the actual environment. Furthermore, the generated layout proposals often do not meet the user's expectations, resulting in the need to make re-proposals, which is time-consuming. Furthermore, it is difficult to confirm whether the proposed layout is actually possible.

[0801] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0802] In this invention, the server includes a means for preprocessing received data, a means for inputting the preprocessed data into a generative AI model, and a means for returning the generated layout proposal to the user. This makes it possible to accommodate a variety of data formats input by users and to provide specific layout proposals that are suited to the actual environment. Furthermore, even if the generated layout proposal does not meet the user's expectations, a new proposal can be quickly made. Furthermore, it becomes easy to confirm whether the proposed layout can actually be arranged.

[0803] "Generative AI means" refers to artificial intelligence technology that learns various room layouts and generates optimal layout suggestions based on input data.

[0804] "Chat-based image input means" refers to an interface that allows users to input images and requests in text format.

[0805] "Means for inputting real-world data" refers to an interface that allows users to upload real-world data, such as LiDAR data, images, and videos of the target room, to the system.

[0806] "Means for generating / proposing layouts" refers to the function in which the AI ​​generating means generates the optimal room layout based on the input real-world environment data and proposes it to the user.

[0807] "Means for outputting product information" refers to a function that provides the user with information about the furniture and home appliances to be placed based on the generated layout.

[0808] "Means for preprocessing received data" refers to the process of analyzing data such as LiDAR data, images, and videos received from users and extracting necessary information.

[0809] "Means for inputting preprocessed data into a generative AI model" refers to the process of inputting preprocessed data into a generative AI means to generate optimal layout proposals.

[0810] "Means for returning the generated layout proposals to the user" refers to an interface for providing the layout proposals generated by the generating AI means to the user.

[0811] This invention is a system that allows a user to input real-world environment data of a target room and generates an optimal layout proposal using a generative AI model. Specific embodiments of this system are described below.

[0812] First, the user inputs real-world data from the target room into the system. This data includes 3D data acquired using a LiDAR scanner, images and videos taken with a smartphone, etc. This data is uploaded from the user's device through the system interface.

[0813] The server receives data uploaded by users. The received data may be in the form of LiDAR data, images, videos, etc. The server first preprocesses this data. Specifically, it converts the LiDAR data into a 3D model and extracts room features from the images and videos. For example, it extracts information such as wall color, furniture placement, and window positions.

[0814] The server then inputs the preprocessed data into a generative AI model, which uses advanced AI techniques such as OpenAI's GPT-4 or DALL-E, to generate an optimal layout based on the input data.

[0815] The server generates specific layout proposals based on the analysis results obtained from the generative AI model. These proposals include furniture placement, design ideas, color combinations, and more. The generated layout proposals are returned to the user. The user can review the proposals through the system interface and request revisions or re-proposals as needed.

[0816] As a concrete example, consider a user who wants to improve the layout of their living room. The user uses a LiDAR scanner to obtain 3D data of the living room and takes photos and videos of the room with their smartphone. These data are then uploaded to the system. The server first analyzes the LiDAR data and generates a 3D model of the room. It then extracts information such as furniture placement and wall color from the images and videos. Based on this data, a generative AI model generates optimal layout suggestions.

[0817] An example of a specific prompt is, "Please analyze the 3D data and images of the living room and suggest optimal furniture placement and design ideas." In this way, users can receive specific layout suggestions from the generative AI model based on the real-world environment data of the room.

[0818] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0819] Step 1:

[0820] The user inputs real-world data

[0821] The user inputs real-world data of the target room into the system. Specifically, the user acquires 3D data of the room using a LiDAR scanner, takes photos and videos of the room using a smartphone, and uploads this data through the system interface. The input data includes LiDAR data, images, and videos.

[0822] Step 2:

[0823] The server receives the data

[0824] The server receives the data uploaded by the user, which may be in the form of LiDAR data, images, videos, etc. The server temporarily stores this data and prepares it for the next processing step.

[0825] Step 3:

[0826] The server preprocesses the data

[0827] The server preprocesses the received data. Specifically, it converts the LiDAR data into a 3D model and extracts room features from images and videos. For example, it extracts information such as wall color, furniture placement, and window position. The input data are LiDAR data, images, and videos, and the output data are the 3D model and room feature information.

[0828] Step 4:

[0829] The server analyzes the data using a generative AI model

[0830] The server inputs the preprocessed data into a generative AI model, which uses advanced AI technology, for example. The generative AI model generates an optimal layout based on the input data. The input data is a 3D model and room feature information, and the output data is a layout proposal.

[0831] Step 5:

[0832] The server generates a layout proposal

[0833] The server generates specific layout proposals based on the analysis results obtained from the generative AI model. These proposals include furniture placement, design ideas, color combinations, etc. The input data is the analysis results of the generative AI model, and the output data is the specific layout proposals.

[0834] Step 6:

[0835] The server returns the proposal to the user

[0836] The server returns the generated layout proposal to the user. The user can review the proposal through the system interface and request corrections or resubmissions as necessary. The input data is the specific layout proposal, and the output data is the proposal information to the user.

[0837] (Application example 2)

[0838] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0839] Conventional store layout optimization relies heavily on experience and intuition, making it difficult to achieve efficient layouts. In addition, there has been a lack of layout proposals that utilize real-world data, making it difficult to achieve optimal layouts that are in line with the actual store environment.

[0840] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout that suits the real environment, a means for outputting product information of the furniture / appliances to be placed, a means for capturing real-world environment data of the store, a means for analyzing the data using a generation AI model and generating an optimal layout proposal, and a means for displaying the generated layout proposal. This enables efficient layout proposals that are suited to the actual store environment.

[0841] A "generative AI method" is an artificial intelligence method that learns various room layouts and generates the optimal layout based on input data.

[0842] A "chat-based image input means" is a means for users to input images and requests in text format.

[0843] "Means for inputting actual environment data" refers to means for inputting actual environment data of the target room or store into the system.

[0844] The "means for generating / proposing layouts" is a means for generating optimal layouts based on input real-world data and proposing them to the user.

[0845] The "means for outputting product information" is a means for providing detailed product information of the furniture and home appliances to be placed to the user.

[0846] "Means for capturing real-world store environment data" refers to means for obtaining real-world environment data such as LiDAR data, images, and videos within the store.

[0847] "Means for analyzing data using a generative AI model" refers to a means for inputting acquired real-world data into a generative AI model and performing analysis.

[0848] The "means for displaying a layout proposal" is a means for visually displaying the generated layout proposal to the user.

[0849] A system for implementing this invention is configured as follows. First, a user captures real-world data in a store using a smartphone. Specifically, the user acquires 3D data using a LiDAR sensor and takes images and videos using the smartphone's camera. This data is then sent to a server via an application on the smartphone.

[0850] The server uses a generative AI model to analyze the received real-world data. The generative AI model has previously learned about various room layouts and generates optimal layout proposals based on the input data. Specifically, the process is carried out as follows:

[0851] 1. Data capture:

[0852] Users use their smartphones to capture LiDAR data and image data inside the store.

[0853] This data is sent to a server via a smartphone application.

[0854] 2. Data Analysis:

[0855] The server inputs the received LiDAR data and image data into the generative AI model.

[0856] The generative AI model generates optimal layout suggestions based on the input data.

[0857] 3. Generate layout proposals:

[0858] The generative AI model analyzes real-world data and generates optimal layout proposals.

[0859] The generated layout proposal is displayed on the user's smartphone.

[0860] The hardware used includes a smartphone, LiDAR sensor, and smartphone camera, while the software used includes a generative AI model (e.g., GPT-3), an application for data capture, and a server for data analysis.

[0861] As a concrete example, the following prompt sentence is input to the generative AI model:

[0862] Example prompt sentence:

[0863] LiDAR data: [0.5, 1.2, 0.3, ...], Image data: [image_data_as_base64_string]

[0864] By inputting this prompt into a generative AI model, the model generates suggestions for optimizing the store layout, such as "rearrange product placement to improve customer flow" or "place specific products in prominent locations."

[0865] In this way, it becomes possible to propose an efficient layout that is suited to the actual store environment.

[0866] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0867] Step 1:

[0868] Users use their smartphones to capture real-world data in the store. Specifically, they acquire 3D data using a LiDAR sensor and take images and videos using the smartphone's camera. The input data is the LiDAR data and image data, which are then saved in the smartphone's application.

[0869] Step 2:

[0870] The user sends the captured real-world data to the server through a smartphone application. The input data is LiDAR data and image data, which are uploaded to the server via the network. The output is the real-world data stored on the server.

[0871] Step 3:

[0872] The server uses a generative AI model to analyze the received real-world data. Specifically, it inputs LiDAR data and image data into the generative AI model. The input data are LiDAR data and image data, and the generative AI model analyzes them to generate an optimal layout proposal. The output is a layout proposal.

[0873] Step 4:

[0874] The server sends the generated layout proposal to the user's smartphone. The input data is the layout proposal from the generative AI model, which is sent to the user's smartphone via the network. The output is the layout proposal displayed on the user's smartphone.

[0875] Step 5:

[0876] The user checks the generated layout proposal through a smartphone application. The input data is the layout proposal sent from the server, which is displayed on the smartphone screen. The output is the layout proposal that the user checks.

[0877] In this way, it becomes possible to propose an efficient layout that is suited to the actual store environment.

[0878] Example 3

[0879] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0880] Conventional layout proposal systems make it difficult for users to obtain specific product information and to determine whether the proposed layout is suitable for the actual environment. In addition, in commercial facilities, they are unable to propose optimal product placement to encourage customer purchasing behavior.

[0881] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout suited to the real environment, a means for outputting product information of the furniture / appliances to be arranged, a means for acquiring product information from a database based on the generated layout proposal, a means for incorporating the acquired product information into the layout proposal, and a means for returning the final layout proposal to the user. This allows the user to obtain a layout proposal including specific product information, making it easy to determine a layout suitable for the actual environment. Furthermore, in commercial facilities, optimal product placement can be proposed to promote customer purchasing behavior.

[0882] "Generative AI means" refers to artificial intelligence that learns various room layouts and generates optimal layout suggestions based on the user's requirements.

[0883] "Chat-based image input means" refers to an interface that allows users to input layout preferences and conditions in text format.

[0884] "Means for inputting real-world data" refers to a means for inputting real-world environmental data, such as the dimensions and shape of the target room and the existing furniture layout, into the system.

[0885] "Means for generating / proposing layouts" refers to means for generating an optimal room layout based on input data and proposing it to the user.

[0886] "Means for outputting product information" refers to means for providing the user with specific product information (manufacturer name, product name, price, etc.) of the furniture and home appliances included in the generated layout.

[0887] The "means for acquiring product information from a database" refers to a means for searching and acquiring necessary product information from a database based on the generated layout proposal.

[0888] The "means for incorporating product information into a layout proposal" refers to a means for integrating acquired product information into a generated layout proposal and providing it to a user.

[0889] The "means for returning a final layout proposal to the user" refers to a means for transmitting the final generated layout proposal to the user.

[0890] This invention is a system for allowing a user to receive room layout proposals, and generates layout proposals including specific product information using a generative AI model. A specific embodiment of this system is described below.

[0891] Server Roles

[0892] The server receives requests from users and generates layout proposals using generative AI models. The server uses the following hardware and software:

[0893] Hardware: High-performance server machine (e.g., CPU, memory, storage)

[0894] Software: Generative AI model, database management system, API server

[0895] The server receives a request from the user, generates and inputs a prompt to the generative AI model, analyzes the generated layout proposal, and retrieves the necessary product information from the database. The retrieved product information is then incorporated into the layout proposal and sent back to the user.

[0896] Device Role

[0897] The device sends user-entered requests to the server and displays layout suggestions returned by the server. The device uses the following hardware and software:

[0898] Hardware: PCs, smartphones, tablets

[0899] Software: Web browser, dedicated application

[0900] The terminal provides an interface for the user to input requests, transmits the requests to the server, receives layout proposals returned from the server, and displays them to the user.

[0901] User Roles

[0902] Users can request a layout proposal through their device, review the returned proposal, and use the specific product information included in the proposal to help them select furniture and home appliances.

[0903] Specific examples

[0904] Here are some example prompts to input to the generative AI model:

[0905] "Please suggest a living room layout that meets the following criteria:

[0906] Includes sofa, TV and coffee table

[0907] Modern design

[0908] The budget is under 200,000 yen.

[0909] When this prompt is sent to the server, the generative AI model generates a living room layout proposal and returns it, including specific product information (e.g., manufacturer name, product name, price, etc.). The user can use this information to select furniture and appliances.

[0910] In this way, the user can obtain layout proposals including specific product information, and can easily determine a layout suitable for the actual environment. Also, in commercial facilities, the system can propose optimal product placement to promote customer purchasing behavior. The flow of the specification process in the third embodiment will be described with reference to FIG. 15.

[0911] Step 1: Receiving a user request

[0912] The server receives a layout proposal request from the user. Specifically, it receives an HTTP request and analyzes the request content. The input is the request data sent by the user from the terminal, and the output is the analyzed request content.

[0913] Step 2: Generate a prompt statement

[0914] The server generates a prompt sentence to be input to the generative AI model based on the user's request. Specifically, it uses a template to construct a prompt sentence based on the user's input data. The input is the parsed request content, and the output is the generated prompt sentence.

[0915] Step 3: Input to the generative AI model

[0916] The server inputs the generated prompt sentence into the generative AI model and generates a layout proposal. Specifically, it calls the API of the generative AI model and sends the prompt sentence. The input is the generated prompt sentence, and the output is the layout proposal returned from the generative AI model.

[0917] Step 4: Get product information

[0918] The server retrieves the necessary product information from a database based on the generated layout proposal. Specifically, it executes a database query to retrieve information such as the manufacturer, product name, and price of the furniture or home appliance. The input is the generated layout proposal, and the output is the retrieved product information.

[0919] Step 5: Assembling the layout proposal

[0920] The server incorporates the acquired product information into the layout proposal to create the final proposal. Specifically, it integrates the generated layout proposal with the product information and constructs data to be sent back to the user. The input is the acquired product information, and the output is the final layout proposal.

[0921] Step 6: Return to user

[0922] The server returns the final layout proposal to the user. Specifically, it generates an HTTP response and sends it to the user's device. The input is the final layout proposal, and the output is the layout proposal sent to the user's device.

[0923] Step 7: Providing a Request Interface

[0924] The terminal provides an interface for the user to request a layout proposal. Specifically, it displays a form and input fields to allow the user to enter conditions. The input is the user's operation, and the output is the request data entered by the user.

[0925] Step 8: Submitting the Request

[0926] The terminal sends the request entered by the user to the server. Specifically, when the user submits a form, an HTTP request is sent to the server. The input is the request data entered by the user, and the output is the request sent to the server.

[0927] Step 9: Receiving the Response

[0928] The terminal receives the layout proposal returned from the server. Specifically, it receives the HTTP response and analyzes the data. The input is the layout proposal returned from the server, and the output is the analyzed layout proposal.

[0929] Step 10: View layout suggestions

[0930] The terminal displays the received layout proposal to the user. Specifically, the layout proposal and product information are displayed on the screen so that the user can confirm them. The input is the analyzed layout proposal, and the output is the layout proposal displayed to the user.

[0931] Step 11: Enter your request

[0932] The user requests a layout proposal through the terminal interface. Specifically, the user enters the conditions into a form and clicks the submit button. The input is the user's desired conditions, and the output is the request data entered into the terminal.

[0933] Step 12: Review the proposal

[0934] The user checks the layout proposal returned from the server. Specifically, the user views the layout proposal and product information displayed on the terminal. The input is the layout proposal displayed on the terminal, and the output is the user's confirmation result.

[0935] Step 13: Product Selection Recommendations

[0936] The user selects furniture or home appliances based on the specific product information included in the proposal. Specifically, the user decides which furniture or home appliance to purchase based on the proposed product information. The input is the proposed product information, and the output is the user's selection.

[0937] (Application example 3)

[0938] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0939] Product placement in traditional stores often relied on experience and intuition, making it difficult to find the optimal placement. Furthermore, there was a lack of means to provide specific product information, leaving users with a lack of reference information to use when making product selections. Furthermore, it was not possible to incorporate a scientific approach based on behavioral economics, making it difficult to effectively promote customer purchasing behavior.

[0940] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0941] In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from the user, a means for inputting actual environment data of the target room, a means for generating / proposing a layout that suits the actual environment, a means for outputting product information for the furniture / appliances to be arranged, a means for proposing product arrangements based on behavioral economics, a means for providing a smartphone application for optimizing product arrangements in a store, and a means for providing specific product information (manufacturer name, product name, price, etc.).By scientifically optimizing product arrangements in a store and providing specific product information, it is possible to enrich reference information for users when making product selections and effectively promote customer purchasing behavior.

[0942] A "generative AI means" is an artificial intelligence means that learns various room layouts and generates the optimal layout based on the user's requirements.

[0943] A "chat-based image input means" is a means for users to input images and requests in a chat format.

[0944] The "means for inputting actual environment data" refers to a means for inputting actual environment data of the target room or store.

[0945] The "means for generating / proposing a layout" is a means for generating and proposing an optimal layout based on input data.

[0946] The "means for outputting furniture / home appliance product information" is a means for outputting specific product information (manufacturer name, product name, price, etc.) of the furniture or home appliance to be arranged.

[0947] "Means for proposing product placement based on behavioral economics" refers to means for proposing optimal product placement that promotes customer purchasing behavior based on the theory of behavioral economics.

[0948] The "means for providing a smartphone application" is a means for providing an application on a smartphone for optimizing product placement in a store.

[0949] The "means for providing specific product information" refers to a means for providing specific product information such as the manufacturer name, product name, price, etc. to the user.

[0950] To implement the present invention, the following system configuration and processing procedures are required.

[0951] First, the server is equipped with a generation AI means that has learned various room layouts. This generation AI means is an artificial intelligence that generates the optimal layout based on past layout data. Users can input their desired layout and image using a chat-based image input means. This communicates the user's specific requests to the system.

[0952] Next, the server is equipped with a means for inputting the actual environment data of the target room or store. This means inputs data such as the dimensions and layout of the actual room or store. Based on this data, the generation AI means generates and proposes a layout that suits the actual environment.

[0953] Furthermore, the server is equipped with a means for outputting product information for the furniture and home appliances to be placed. This means allows the user to check specific product information (manufacturer name, product name, price, etc.), which can be used as a reference for the user in selecting specific products.

[0954] The server also has a means for proposing product placement based on behavioral economics. This allows it to propose optimal product placement that encourages customer purchasing behavior, thereby scientifically optimizing product placement within the store.

[0955] Furthermore, the server includes a means for providing a smartphone application for optimizing product placement in a store. By using this application, the store manager or staff can optimize product placement using their smartphones.

[0956] The server also has a means for providing specific product information, which allows the user to obtain specific product information such as the manufacturer's name, product name, price, etc.

[0957] The hardware used is a smartphone, and the software uses Python and the OpenAI API. Data processing involves formatting store data (product list, current layout information) in JSON format, and then using a generative AI model to generate the optimal product placement.

[0958] As a concrete example, by inputting the following prompt sentence into the generative AI model, optimal product placement suggestions can be obtained.

[0959] Example prompt sentence:

[0960] Store data: {"Store name": "ABC Store", "Product list": [{"Manufacturer name": "Manufacturer A", "Product name": "Product A", "Price": 1000}, {"Manufacturer name": "Manufacturer B", "Product name": "Product B", "Price": 2000}], "Store layout": "Current layout information"}

[0961] Please suggest optimal product placement based on behavioral economics.

[0962] In this way, the server, terminal, and user can work together to achieve optimal product placement.

[0963] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0964] Step 1:

[0965] The user starts the application on their smartphone and inputs their desired layout and image using a chat-based image input means.

[0966] Input: The layout and image desired by the user

[0967] Output: User's desired layout and image data

[0968] Step 2:

[0969] The server receives the image data sent from the user and inputs data such as the dimensions and layout of the actual room or store using a means for inputting real environment data of the target room or store.

[0970] Input: User image data, real-world data

[0971] Output: Integrated layout data

[0972] Step 3:

[0973] The server uses a generation AI means to generate an optimal layout based on the integrated layout data. The generation AI means learns from past layout data and proposes an optimal layout taking into account the user's requests and actual environment data.

[0974] Input: Integrated layout data

[0975] Output: Optimal layout proposal

[0976] Step 4:

[0977] The server provides the user with specific product information (manufacturer name, product name, price, etc.) for the furniture and home appliances to be arranged based on the generated layout proposal, using a means for outputting product information.

[0978] Input: Optimal layout proposal

[0979] Output: Specific product information

[0980] Step 5:

[0981] The server uses a method based on behavioral economics to suggest optimal product placements that encourage customer purchasing behavior, thereby scientifically optimizing product placements within the store.

[0982] Input: Optimal layout proposals, behavioral economics data

[0983] Output: Optimal product placement proposal

[0984] Step 6:

[0985] The server notifies the store manager and staff of the optimal product placement using a means for providing a smartphone application for optimizing product placement within the store.

[0986] Input: Optimal product placement proposal

[0987] Output: Notification on smartphone application

[0988] Step 7:

[0989] Users can use the smartphone application to check specific product information and use it as a reference for product selection.

[0990] Input: Specific product information

[0991] Output: User selected product information

[0992] In this way, the server, terminal, and user can work together to achieve optimal product placement.

[0993] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0994] "Example 1"

[0995] One embodiment of the present invention provides a system incorporating an emotion engine. This system recognizes a user's emotions and adjusts room layout suggestions based on those emotions. Specifically, if a user inputs that they desire a "relaxed living room," the emotion engine recognizes the user's current emotional state (e.g., relaxed, tense, etc.). It then adjusts the layout suggestions to match the user's emotional state, for example, by suggesting furniture with more subdued colors if the user is relaxed, or by suggesting furniture arrangements that create a relaxing effect if the user is tense.

[0996] "Example 2"

[0997] The emotion engine is also used to suggest product choices. For example, if a user inputs "I'm looking for a new TV," the emotion engine will recognize the user's current emotional state and suggest product choices based on that emotional state. For example, if the user is feeling happy, it will suggest a large-screen, high-end TV, and if the user is feeling sad, it will suggest a cost-effective TV.

[0998] "Example 3"

[0999] Furthermore, the emotion engine learns changes in the user's emotions and optimizes layout suggestions and product selection suggestions based on the results of that learning. For example, if a user feels a strong sense of joy when looking at a certain product, the engine will prioritize suggestions for that product or products in the same category. The engine learns how users respond to emotions and makes suggestions that reflect this.

[1000] "Example 4"

[1001] The emotion engine also analyzes users' emotional responses and makes product placement suggestions based on the results. For example, if a user expresses joy at the placement of a certain product shelf, the engine will apply that same placement to other product shelves.

[1002] The processing flow of each embodiment will be described below.

[1003] "Example 1"

[1004] Step 1: The user inputs their desired room layout into the system.

[1005] Step 2: The emotion engine recognizes the user's current emotional state.

[1006] Step 3: Adjust the layout proposal based on the emotional state recognized by the emotion engine.

[1007] "Example 2"

[1008] Step 1: The user enters their product selection preferences into the system.

[1009] Step 2: The emotion engine recognizes the user's current emotional state.

[1010] Step 3: Tailor product selection suggestions based on the emotional state recognized by the emotion engine.

[1011] "Example 3"

[1012] Step 1: The emotion engine learns the user's emotional changes.

[1013] Step 2: Based on the results learned by the emotion engine, layout suggestions and product selection suggestions are optimized.

[1014] "Example 4"

[1015] Step 1: The emotion engine analyzes the user's emotional response.

[1016] Step 2: Based on the results of the emotion engine's analysis, product placement suggestions are made.

[1017] Example 1

[1018] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1019] Conventional room layout suggestion systems make uniform suggestions without considering the user's emotional state, making it difficult to provide a layout that suits the user's current psychological state. In addition, layouts generated based on user input may not match the actual environment or emotional state, resulting in a decrease in user satisfaction.

[1020] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout suited to the real environment, a means for outputting product information of furniture / appliances to be placed, an emotion engine means for recognizing the emotional state of the user, and a means for adjusting the layout proposal based on the emotional state. This makes it possible to propose a layout suited to the emotional state of the user, thereby improving user satisfaction.

[1021] "Generative AI means" is an artificial intelligence technology that learns various room layouts and generates new layout suggestions based on user input.

[1022] The "chat-based image input means" is an interface that allows users to input their image and preferences for the room in text format.

[1023] The "means for inputting real environment data" refers to a means for inputting environmental data such as the actual dimensions and layout of the target room into the system.

[1024] "Means for generating / proposing layouts" refers to means for proposing layouts generated by the generation AI means to users.

[1025] The "means for outputting furniture / home appliance product information" is a means for providing the user with detailed information about the furniture and home appliances that will be arranged based on the proposed layout.

[1026] "Emotion engine means" is a technology for recognizing the current emotional state of a user.

[1027] The "means for adjusting layout proposals based on emotional state" refers to means for optimizing the generated layout proposals in response to the perceived emotional state of the user.

[1028] This invention is a system in which a user inputs a room image through a chat-based interface, and a generative AI model makes layout suggestions based on that input. Furthermore, by combining it with an emotion engine, it provides optimal layout suggestions according to the user's emotional state.

[1029] The server uses the following hardware and software to realize this system: For the generative AI model, it uses OpenAI's GPT-4, for example. For the emotion engine, it uses Affectiva's emotion recognition API, for example. By combining these technologies, it generates layout proposals based on the user's input and emotional state.

[1030] Using a chat-based interface (e.g., a web application or mobile app), a user inputs an image of the room they want, such as "a simple, calm living room." This input is sent to a generative AI model running on a server.

[1031] The generative AI model analyzes user input and combines it with previously learned room layout data to generate initial layout suggestions. For example, for a "simple and calm living room," it suggests wall colors, furniture placement, lighting types, etc.

[1032] The server then uses an emotion engine to recognize the user's current emotional state: if the user provides emotion data through the camera or microphone, the emotion engine analyzes that data to determine whether the user is relaxed or tense.

[1033] The server adjusts the generative AI model's initial layout proposals based on the emotional state obtained from the emotion engine. For example, if the user is relaxed, it will suggest furniture with more muted colors, and if the user is tense, it will suggest furniture arrangements that have a relaxing effect.

[1034] As a concrete example, consider a case where a user inputs that they want a "simple and relaxing living room." If the emotion engine recognizes that the user is relaxed, the generative AI model will generate the following layout suggestions:

[1035] The wall color is light blue

[1036] The sofa is gray

[1037] The lighting is warm-colored indirect lighting.

[1038] On the other hand, if the emotion engine detects that the user is nervous, the generative AI model will generate layout suggestions like this:

[1039] The wall color is pale green

[1040] The sofa is beige

[1041] The lighting is soft white light

[1042] Example prompt sentence:

[1043] User input: "A simple, calm living room."

[1044] Emotional state: Relaxed

[1045] Output of the generative AI model: "The walls are pale blue, the sofa is gray, and the lighting is warm, indirect lighting."

[1046] In this way, the server can generate and provide optimal room layout suggestions to the user based on the user's input and emotional state.

[1047] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1048] Step 1:

[1049] The user inputs the image of the room through a chat-based interface. The user then enters specific details in text format, such as "a living room with a simple and calm atmosphere." This input becomes the initial data for the system.

[1050] Input: Image of the user's room (e.g., "A simple, calm living room")

[1051] Output: Image data of the user's desired room

[1052] Step 2:

[1053] The server receives the user's input and sends it to the generative AI model. The server converts the user's input data into an appropriate format so that the generative AI model can analyze it.

[1054] Input: Image data of the room desired by the user

[1055] Output: Parsable data sent to a generative AI model

[1056] Step 3:

[1057] A generative AI model analyzes user input and generates initial layout suggestions. The model combines learned room layout data with user input to suggest wall colors, furniture placement, lighting types, and more.

[1058] Input: Parsable user image data

[1059] Output: Initial layout suggestions (e.g. wall color, furniture placement, lighting type)

[1060] Step 4:

[1061] The server uses an emotion engine to recognize the user's current emotional state. When the user provides emotion data through a camera or microphone, the emotion engine analyzes the data to determine whether the user is relaxed or tense.

[1062] Input: User emotion data (e.g. camera footage, audio data)

[1063] Output: User's emotional state (e.g. relaxed, tense)

[1064] Step 5:

[1065] The server adjusts the layout proposal of the generative AI model based on the emotional state. The server optimizes the initial layout proposal of the generative AI model based on the emotional state obtained from the emotion engine. For example, if the user is relaxed, it will suggest furniture with more subdued colors, and if the user is tense, it will suggest furniture arrangements that have a relaxing effect.

[1066] Input: initial layout proposal, user emotional state

[1067] Output: Final, adjusted layout proposal

[1068] Step 6:

[1069] The server sends the final layout proposal back to the user, who can review it through a chat-based interface and provide feedback if desired.

[1070] Input: Final tweaked layout proposal

[1071] Output: Present layout proposal to user

[1072] In this way, the server can generate and provide optimal room layout suggestions to the user based on the user's input and emotional state.

[1073] (Application example 1)

[1074] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1075] Conventional interior design proposal systems do not consider the user's emotional state when proposing layouts, making it difficult to increase the user's psychological satisfaction. Furthermore, in physical stores, there is a demand for optimal interior designs based on the user's emotions, but there has been a lack of effective means to achieve this.

[1076] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from the user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout that suits the real-world environment, a means for outputting product information of the furniture / appliances to be placed, an emotion engine means for recognizing the user's emotional state, and a means for adjusting the layout proposal based on the user's emotional state. This makes it possible to propose an optimal interior design that takes the user's emotional state into consideration.

[1077] "Generative AI means" is an artificial intelligence technology that learns various room layouts and generates new layout suggestions based on user input.

[1078] The "chat-based image input means" is an interface that allows users to input their desired image of the room in a chat format.

[1079] The "means for inputting real environment data" refers to a means for inputting the actual environment data of the target room (e.g., room dimensions, furniture arrangement, etc.).

[1080] The "means for generating / proposing layouts" is a means for generating a room layout suitable for the actual environment based on input data and proposing it to the user.

[1081] The "means for outputting product information" is a means for providing the user with information about the furniture and home appliances to be placed in the generated layout.

[1082] "Emotion engine means" is a technology for recognizing the user's emotional state and adjusting the system's behavior based on that information.

[1083] The "means for adjusting layout proposals based on emotional state" refers to a means for optimizing layout proposals by taking into account the emotional state of the user.

[1084] The following system configuration will be described as an embodiment of the present invention.

[1085] First, the server is equipped with a generation AI means that has learned various room layouts. This generation AI means analyzes the image of the desired room input by the user through a chat-based image input means, and combines it with the learned data to generate new layout proposals.

[1086] Users use their smartphones to chat with the system to input their desired image of the room. For example, they can input specific requests, such as "a relaxing cafe-style interior." This input is sent to the server and analyzed by the AI ​​generator.

[1087] Next, the server is provided with a means for inputting real-world environment data of the target room, such as the dimensions of the room and the existing furniture arrangement, provided by the user, which is used by the generative AI means to suggest an optimal layout.

[1088] Furthermore, the server is equipped with a means for generating and proposing layouts suited to the actual environment. Based on the user's preferences and actual environment data analyzed by the generation AI means, the server generates and proposes an optimal layout to the user. This proposal is displayed to the user via a smartphone application.

[1089] The server also has a means for outputting product information for the furniture and home appliances to be placed. Based on the generated layout, detailed information on the proposed furniture and home appliances is provided to the user, allowing the user to easily purchase the products required for the proposed layout.

[1090] Furthermore, the server is equipped with an emotion engine means for recognizing the user's emotional state. This emotion engine means analyzes the user's facial expressions and tone of voice to recognize the user's current emotional state. For example, it can determine whether the user is tense or relaxed.

[1091] Finally, the server includes a means for adjusting the layout proposal based on the user's emotional state. The generative AI means adjusts the layout proposed based on the user's emotional state recognized by the emotion engine means. For example, if the user is feeling tense, the server suggests colors and furniture arrangements that have a relaxing effect.

[1092] For example, if a user requests a "relaxing cafe-style interior" and their emotional state is recognized as "tense," the AI ​​generator will suggest colors and furniture arrangements that have a relaxing effect. An example of the prompt sentence in this case is as follows:

[1093] Desired interior design: A relaxing cafe-style interior

[1094] User's emotional state: nervous

[1095] Propose the best interior design for you.

[1096] In this way, it is possible to propose optimal interior designs based on the user's wishes and emotional state.

[1097] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1098] Step 1:

[1099] The user uses a smartphone application to input the image of the room they want in chat format. For example, they might input "a relaxing cafe-style interior." This input data is then sent to the server.

[1100] Step 2:

[1101] The server receives image input data sent by the user, analyzes the received data, and generates a prompt sentence to input into the generative AI model. Specifically, it creates a prompt sentence containing the user's desired interior design.

[1102] Step 3:

[1103] The server uses the generative AI model to generate layout proposals based on the prompt sentence. The generative AI model generates a new layout that meets the user's wishes based on the learned room layout data. This generated layout proposal is saved on the server.

[1104] Step 4:

[1105] The user inputs the real-world environment data of the target room (e.g., room dimensions, existing furniture layout, etc.) through a smartphone application, and this data is also sent to the server.

[1106] Step 5:

[1107] The server receives the real-world data sent by the user, analyzes the received data, and combines it with the generated layout proposal to generate an optimal layout suited to the real-world environment. This optimized layout proposal is then stored in the server.

[1108] Step 6:

[1109] The server uses an emotion engine means to recognize the user's emotional state. It analyzes the user's facial expressions and tone of voice to recognize the user's current emotional state. For example, it determines whether the user is tense or relaxed.

[1110] Step 7:

[1111] The server adjusts the generated layout proposal based on the user's emotional state recognized by the emotion engine means. For example, if the user is nervous, the server may suggest colors and furniture arrangements that have a relaxing effect. The adjusted layout proposal is stored in the server.

[1112] Step 8:

[1113] The server generates product information for the furniture and home appliances to be arranged based on the adjusted layout proposal, and transmits the generated product information to the smartphone application to be provided to the user.

[1114] Step 9:

[1115] The user receives the adjusted layout proposal and product information sent from the server through a smartphone application, allowing the user to check and purchase the products required for the proposed layout.

[1116] In this way, an optimal interior design proposal based on the user's wishes and emotional state is realized.

[1117] Example 2

[1118] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1119] Conventional room layout suggestion systems make uniform suggestions without considering the user's emotional state, which fails to sufficiently increase user satisfaction. Furthermore, they lack the functionality to analyze real-world data to suggest optimal layouts, making it difficult to provide specific suggestions tailored to the user's needs. Furthermore, product selection suggestions do not reflect the user's emotional state, failing to stimulate the user's purchasing motivation.

[1120] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1121] In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from the user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout that suits the real environment, a means for outputting product information of the furniture / appliances to be placed, an emotion engine means for recognizing the emotional state of the user, and a means for suggesting product selection based on the emotional state. This makes it possible to propose optimal layouts and product selections that take the user's emotional state into consideration.

[1122] "Generative AI means" is an artificial intelligence technology that learns various room layouts and generates the optimal layout for the user.

[1123] A "chat-based image input means" is an interface that allows users to input images and requests in text format.

[1124] "Means for inputting real-world data" refers to the means for inputting real-world data such as LiDAR data, images, and videos of the target room into the system.

[1125] "Means for generating / proposing layouts suited to the actual environment" refers to means for generating and proposing layouts that are optimal for users based on inputted actual environment data.

[1126] The "means for outputting product information of the furniture / home appliances to be arranged" is a means for providing the user with detailed information about the furniture and home appliances to be arranged based on the proposed layout.

[1127] "Emotion engine means" is a technology for recognizing the current emotional state of a user.

[1128] The "means for suggesting product selection based on emotional state" is a means for suggesting optimal products according to the recognized emotional state of the user.

[1129] This invention is a system for users to optimize their room layout, and it combines a generative AI model and an emotion engine to make product recommendations based on the user's emotional state. Specific embodiments of this system are described below.

[1130] 1. Data Entry

[1131] Users access the system interface using a smartphone or PC, select real-world data such as LiDAR data, images, and videos, and click the upload button. This allows users to input the real-world data of the target room into the system.

[1132] 2. Data transmission

[1133] The terminal sends the data selected by the user to the server using a secure protocol (e.g., HTTPS). When the transmission is complete, the terminal notifies the user that "Data transmission is complete."

[1134] 3. Data Analysis

[1135] The server analyzes the received data. The LiDAR data is used to reconstruct the dimensions and shape of the room as a 3D model using OpenCV. Images and videos are used to recognize colors and furniture placement. The analysis results are stored in an internal database.

[1136] 4. Generating layout proposals

[1137] The server generates layout proposals using a generative AI model (e.g., GPT-4 or DALL-E) based on the analysis results. The proposals are sent to the user's device and displayed on the interface. The user can review the proposed layout and make any necessary modifications.

[1138] 5. Emotion Recognition by Emotion Engine

[1139] When a user types "I'm looking for a new TV," the device sends this information to a server, which uses the Emotion API to analyze the user's facial images and voice data to recognize their current emotional state.

[1140] 6. Emotion-based product recommendations

[1141] The server then suggests product choices based on the results of the emotion engine. For example, if the user is feeling happy, it might suggest the latest large-screen 4K TV, and if they are feeling sad, it might suggest a cost-effective 32-inch TV. The suggestions are sent to the user's device and displayed on the interface.

[1142] Examples of concrete examples and prompts

[1143] As a concrete example, consider a case where a user wants to change the layout of their room. The user inputs LiDAR data and images of the room into the system. The server analyzes this data and uses a generative AI model to suggest new furniture placements. Furthermore, if the user inputs "I'm looking for a new TV," the emotion engine recognizes the user's emotional state and suggests an appropriate TV.

[1144] Example prompt sentence:

[1145] "I would like to change the layout of my room, so please analyze the following data and suggest a new layout. I am attaching LiDAR data and room images. Also, I am looking for a new TV, so please suggest a TV based on my emotional state."

[1146] In this way, the system works in cooperation with the user, terminal, and server, and can propose optimal layouts and products to the user.

[1147] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1148] Step 1: Data entry

[1149] Users access the system's interface using a smartphone or PC. On the interface, they select real-world data, such as LiDAR data, images, or videos, and click the upload button. This allows the user to input the real-world data of the target room into the system. The input data includes the room's dimensions, shape, color, furniture arrangement, etc.

[1150] Step 2: Sending data

[1151] The terminal sends the data selected by the user to the server using a secure protocol (e.g., HTTPS). Once the transmission is complete, the terminal notifies the user that "Data transmission is complete." The input data is then sent to the server and prepared for analysis.

[1152] Step 3: Analyze the data

[1153] The server analyzes the received data. The LiDAR data is used to reconstruct the dimensions and shape of the room as a 3D model using OpenCV. Images and videos are used to recognize colors and furniture placement using TensorFlow. The analysis results are stored in an internal database and used in the next step. The input data is output as a 3D model and recognition results.

[1154] Step 4: Generate layout proposals

[1155] The server generates layout proposals using a generative AI model (e.g., GPT-4 or DALL-E) based on the analysis results. The generated proposals are sent to the user's device and displayed on the interface. The user can review the proposed layout and make corrections as necessary. The input data is the analysis results, and the output data is the layout proposals.

[1156] Step 5: Emotion Recognition with the Emotion Engine

[1157] When a user types "I'm looking for a new TV," the device sends this information to the server. The server uses the Emotion API to analyze the user's facial image and voice data to recognize their current emotional state. The input data is the user's text input and facial image and voice data, and the output data is their emotional state.

[1158] Step 6: Emotion-based product recommendations

[1159] The server then suggests product selections based on the results of the emotion engine. For example, if the user is feeling happy, it suggests the latest large-screen 4K TV, and if they are feeling sad, it suggests a cost-effective 32-inch TV. The suggestions are sent to the user's device and displayed on the interface. The input data is the emotional state, and the output data is the product suggestions.

[1160] (Application example 2)

[1161] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1162] Conventional layout suggestion systems simply make layout suggestions based on real-world data without taking into account the user's emotional state. As a result, they are unable to make optimal product or layout suggestions based on the user's emotional or psychological state, and this does not sufficiently improve the user experience. Furthermore, there has been no system that can suggest optimal product placement or layouts in a physical store. This makes it difficult to maximize users' purchasing motivation.

[1163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1164] In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout that suits the real environment, a means for outputting product information for furniture / appliances to be placed, an emotion engine means for analyzing the emotional state of the user, a means for proposing products based on the emotional state, and a means for proposing optimal product placement and layout in a physical store. This makes it possible to propose optimal products and layouts according to the user's emotional state, thereby improving the shopping experience in a physical store.

[1165] "Generative AI means" is an artificial intelligence technology that learns various room layouts and generates layouts based on the real environment.

[1166] A "chat-based image input means" is an interface that allows users to input images and requests in text format.

[1167] "Means for inputting real-world data" refers to the means for incorporating actual data (e.g., LiDAR data, images, videos) of the target room or space into the system.

[1168] The "means for generating / proposing layouts" is a means for generating an optimal layout based on input real-world data and proposing it to the user.

[1169] The "means for outputting product information" is a means for providing the user with information about the furniture and home appliances to be placed in the generated layout.

[1170] The "emotion engine means" is a technology for analyzing the user's emotional state and making appropriate suggestions based on that state.

[1171] "Means for suggesting products" refers to means for selecting and suggesting optimal products based on the user's emotional state.

[1172] "Means for proposing optimal product placement and layout within a physical store" refers to a means for analyzing environmental data within a physical store and proposing optimal product placement and layout.

[1173] A system for carrying out this invention has the following configuration: A server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout that suits the real environment, a means for outputting product information for furniture / home appliances to be placed, an emotion engine means for analyzing the emotional state of the user, a means for proposing products based on the emotional state, and a means for proposing optimal product placement and layout in a physical store.

[1174] Hardware and Software Configuration

[1175] Hardware:

[1176] Smartphone camera

[1177] A LiDAR sensor connected to a smartphone (or a LiDAR built into the smartphone)

[1178] software:

[1179] OpenCV (camera data acquisition and processing)

[1180] EmotionRecognizer (a custom library for emotion recognition)

[1181] LayoutGenerator (a custom library for generating layouts)

[1182] ProductRecommender (a custom library for product recommendations)

[1183] System Operation

[1184] As users walk around the store with their smartphones, they use the camera and LiDAR sensor to collect real-world data. The smartphone camera captures video in real time and processes the image data using OpenCV. The LiDAR sensor then captures 3D data of the target room or space.

[1185] The server analyzes the captured camera data using EmotionRecognizer to recognize the user's emotional state. For example, if the user is smiling, it is recognized as "joy," and if they are frowning, it is recognized as "sadness."

[1186] Next, the server analyzes the acquired LiDAR data using LayoutGenerator and generates a layout that is optimal for the real environment. The generated layout is displayed on the user's smartphone, allowing the user to check the proposal.

[1187] Furthermore, the server uses an emotion engine means to suggest products based on the user's emotional state. For example, if the user is feeling "joy," the server suggests high-quality products, and if the user is feeling "sad," the server suggests cost-effective products.

[1188] Specific examples

[1189] As a concrete example, consider a scenario in which a user walks around a store with their smartphone, and an application makes layout and product suggestions in real time. In this case, the user collects real-world data using the smartphone's camera and LiDAR sensor, and the server analyzes that data to make optimal layout and product suggestions.

[1190] Example prompt sentence:

[1191] "As users walk around the store with their smartphones, they use cameras and LiDAR sensors to collect data on the real-world environment. The application analyzes this data and suggests optimal product placement and layout within the store. It also analyzes the user's emotional state in real time and makes product recommendations based on that emotion."

[1192] In this way, it becomes possible to propose optimal products and layouts according to the user's emotional state, improving the shopping experience in physical stores.

[1193] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1194] Step 1:

[1195] Users walk around the store with their smartphones, and real-world data is collected using cameras and LiDAR sensors.

[1196] Input: Smartphone camera footage and LiDAR sensor data.

[1197] Output: Camera image data and LiDAR 3D data.

[1198] How it works: The smartphone camera captures video in real time, and the LiDAR sensor acquires three-dimensional data of the target room or space.

[1199] Step 2:

[1200] The server receives the camera data and processes the image data using OpenCV.

[1201] Input: Camera video data.

[1202] Output: Processed image data.

[1203] Specific operation: The server receives the camera image data sent from the smartphone and performs image preprocessing (e.g., noise removal, resolution adjustment) using the OpenCV library.

[1204] Step 3:

[1205] The server uses EmotionRecognizer to analyze the user's emotional state from the processed image data.

[1206] Input: Processed image data.

[1207] Output: The user's emotional state (e.g., happy, sad).

[1208] Specific operation: The server uses the EmotionRecognizer library to extract the user's facial features from the processed image data and recognize their emotional state.

[1209] Step 4:

[1210] The server receives the LiDAR data and uses LayoutGenerator to generate a layout that is optimal for the real environment.

[1211] Input: 3D LiDAR data.

[1212] Output: The generated layout proposal.

[1213] Specific operation: The server receives LiDAR data sent from the smartphone and uses the LayoutGenerator library to generate an optimal layout based on the real environment.

[1214] Step 5:

[1215] The server sends the generated layout to the user's smartphone, where the user can check it.

[1216] Input: The generated layout proposal.

[1217] Output: Layout proposals displayed on the user's smartphone.

[1218] Specific operation: The server sends the generated layout proposal to the user's smartphone, and the user checks the proposed layout on the smartphone screen.

[1219] Step 6:

[1220] The server uses an emotion engine means to make product suggestions based on the user's emotional state.

[1221] Input: The user's emotional state.

[1222] Output: Product suggestions based on emotional state.

[1223] Specific operation: The server uses the emotion engine means to select and suggest the most suitable products based on the user's emotional state.

[1224] Step 7:

[1225] The server sends product suggestions to the user's smartphone, which the user confirms.

[1226] Input: Product suggestions based on emotional state.

[1227] Output: Product suggestions displayed on the user's smartphone.

[1228] Specific operation: The server sends product suggestions based on the user's emotional state to the user's smartphone, and the user checks the suggested products on the smartphone screen.

[1229] Example 3

[1230] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1231] Conventional layout suggestion systems have difficulty making optimal suggestions based on user emotions and preferences. Furthermore, they have been unable to make optimal suggestions based on behavioral economics for product and shelf layout in commercial facilities. This has led to the issue of not being able to fully stimulate user satisfaction and purchasing motivation.

[1232] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1233] In this invention, the server includes a generation AI means, a chat-based image input means from the user, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout suited to the real environment, a means for outputting product information of the furniture / appliances to be arranged, an emotion engine means for analyzing the user's emotions, and a means for optimizing the layout proposal based on the analysis results of the emotion engine. This makes it possible to propose an optimal layout based on the user's emotions and preferences, and also to propose product and shelf placements based on behavioral economics.

[1234] "Generative AI methods" refer to artificial intelligence techniques that generate layout suggestions based on user input.

[1235] "Chat-based image input means from users" refers to an interface that allows users to input layout preferences and images in chat format.

[1236] The "means for inputting actual environment data of the target room" refers to a means for inputting actual environment data such as the dimensions and layout of the target room.

[1237] "Means for generating / proposing layouts suited to the actual environment" refers to means for generating and proposing optimal layouts based on inputted actual environment data.

[1238] "Means for outputting product information of furniture / appliances to be arranged" refers to means for outputting specific product information (manufacturer name, product name, price, etc.) of furniture and appliances included in the proposed layout.

[1239] "Emotion engine means for analyzing user emotions" refers to technology for analyzing user feedback and reactions and identifying emotions.

[1240] "Means for optimizing layout proposals based on the analysis results of the emotion engine" refers to means for generating optimal layout proposals based on the analysis results of the emotion engine in accordance with the user's emotions and preferences.

[1241] "Extension function that suggests product / shelf layout methods based on behavioral economics" refers to an extension function that suggests optimal product and shelf layout methods in commercial facilities, etc., based on the theory of behavioral economics.

[1242] This invention is a system that provides users with layout suggestions, including specific product information for furniture and home appliances. It also includes an extension function that makes optimal suggestions based on behavioral economics for product and shelf placement in commercial facilities. The system uses an emotion engine to learn changes in user emotions, and can optimize layout suggestions and product selection suggestions based on the results.

[1243] Hardware and software used

[1244] Hardware: Servers, user devices (PCs, smartphones, tablets)

[1245] Software: Generative AI models (e.g., GPT-4), emotion engines, database management systems (e.g., MySQL), web application frameworks (e.g., Django)

[1246] Specific operation of the system

[1247] 1. Accepting user input

[1248] A user uses a device to input a request for a layout suggestion, for example, "Please suggest a layout for my living room."

[1249] 2. Generating layout proposals using a generative AI model

[1250] The server uses a generative AI model (e.g., GPT-4) to generate layout proposals based on the user's requests, including specific furniture and appliance information (e.g., manufacturer name, product name, price, etc.).

[1251] 3. The server generates layout suggestions and sends them to the user.

[1252] The server sends the layout proposals received from the generative AI model to the user's device, where the user can view the proposals.

[1253] 4. User reviews the proposal and provides emotional feedback

[1254] The user reviews the proposed layout and provides emotional feedback, for example, "I like this sofa." The user's feedback is sent to the server.

[1255] 5. User sentiment analysis using an emotion engine

[1256] The server analyzes the user feedback using an emotion engine, which identifies the user's emotions (happiness, interest, indifference, etc.) and stores the results in a database.

[1257] 6. Emotion-based recommendation optimization

[1258] The server sends a new prompt to the generative AI model based on the analysis results of the emotion engine. The prompt contains optimization instructions based on the user's emotions. The generative AI model then generates an optimized layout proposal.

[1259] 7. The server sends the optimized offer to the user

[1260] The server sends the optimized layout proposal received from the generative AI model to the user's device, where the user can view the optimized proposal.

[1261] Examples of concrete examples and prompts

[1262] Examples:

[1263] User types, "Suggest a living room layout."

[1264] The server uses the generated AI model to generate layout proposals that include specific product information such as "sofa (manufacturer name: general manufacturer, product name: general product, price: 50,000 yen) and television (manufacturer name: general manufacturer, product name: general product, price: 80,000 yen)."

[1265] A user types, "I love this sofa," and the emotion engine learns the emotion of joy.

[1266] Based on the learning results of the emotion engine, the server will prioritize suggestions for sofas and related products in the same category.

[1267] Example prompt sentence:

[1268] "Please suggest a living room layout. Please include specific information about the furniture and appliances (manufacturer, product name, price, etc.)."

[1269] "I love this sofa. Are there any other sofas you'd recommend?"

[1270] In this way, the user can receive an optimal layout proposal based on their emotions while referring to specific product information. The flow of the specification process in the third embodiment will be described with reference to FIG.

[1271] Step 1:

[1272] A user uses a terminal to input a request for a layout proposal. For example, the user may input "Please suggest a layout for the living room." The input request is sent to the server. The server receives the request and proceeds to the next processing step.

[1273] Step 2:

[1274] The server sends a prompt to a generative AI model (e.g., GPT-4). The prompt contains the user's request. The generative AI model generates a layout proposal based on the prompt. The generated layout proposal includes specific furniture and appliance information (manufacturer name, product name, price, etc.). The server receives the output from the generative AI model and proceeds to the next processing step.

[1275] Step 3:

[1276] The server sends the layout proposal received from the generative AI model to the user's device, where the user can review the proposal. After the user has reviewed the proposal, the process proceeds to the next step.

[1277] Step 4:

[1278] The user reviews the proposed layout and provides emotional feedback, for example, "I like this sofa." The feedback is sent to the server, which receives it and proceeds to the next processing step.

[1279] Step 5:

[1280] The server uses the emotion engine to analyze the user's feedback. The emotion engine identifies the user's emotion (happiness, interest, indifference, etc.) and stores the result in a database. The server receives the analysis result from the emotion engine and proceeds to the next processing step.

[1281] Step 6:

[1282] The server sends a new prompt to the generative AI model based on the analysis results of the emotion engine. The prompt contains optimization instructions based on the user's emotions. The generative AI model generates an optimized layout proposal based on the new prompt. The server receives the output from the generative AI model and proceeds to the next processing step.

[1283] Step 7:

[1284] The server sends the optimized layout proposal received from the generative AI model to the user's device, where the user can check the optimized proposal on their device. This allows the user to receive the optimal layout proposal based on their emotions while referring to specific product information.

[1285] (Application example 3)

[1286] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1287] Conventional layout suggestion systems did not take into account user emotions or purchasing history when making suggestions, making it difficult to increase user satisfaction. Furthermore, they lacked the functionality to perform real-time emotion analysis and recommend products based on that analysis, making it difficult to maximize users' purchasing motivation. Furthermore, they lacked product and shelf layout suggestions based on behavioral economics, making it difficult to achieve optimal product placement in commercial facilities.

[1288] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1289] In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from users, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout that suits the real-world environment, a means for outputting product information for the furniture / appliances to be placed, an emotion recognition means for analyzing the user's emotions in real time, a product recommendation means for recommending products based on emotions, and a learning means for learning the user's purchasing history and emotional data and optimizing future proposals. This enables optimal layout proposals and product recommendations that take into account the user's emotions and purchasing history, thereby increasing user satisfaction. Furthermore, by proposing product and shelf placements based on behavioral economics, optimal product placement in commercial facilities can be achieved.

[1290] A "generative AI means" is an artificial intelligence that learns various room layouts and generates the optimal layout based on the user's requirements.

[1291] "Chat-based image input means" refers to an interface that allows users to input images and requests in text format.

[1292] "Means for inputting real-world data" refers to the means for acquiring actual data of the target room or space and inputting it into the system.

[1293] "Means for generating / proposing layouts" refers to means for generating optimal layouts based on input data and proposing them to users.

[1294] The "means for outputting product information" refers to a means for providing the user with specific product information about the furniture and home appliances to be placed.

[1295] An "emotion recognition means" is a means for analyzing a user's emotions in real time and reflecting the results in the system.

[1296] A "product recommendation means" is a means for recommending optimal products based on the user's emotions.

[1297] "Learning methods" are means for learning from a user's purchasing history and emotional data and optimizing the next suggestion.

[1298] A system for carrying out this invention has the following configuration: The server includes a generation AI means that has learned various room layouts, a chat-based image input means from the user, a means for inputting real-world environmental data of the target room, a means for generating / proposing a layout that suits the real-world environment, a means for outputting product information about the furniture / home appliances to be placed, an emotion recognition means that analyzes user emotions in real time, a product recommendation means that recommends products based on emotions, and a learning means that learns the user's purchasing history and emotional data and optimizes the next proposal.

[1299] Hardware and software used

[1300] Hardware: Smartphone camera, server

[1301] Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition model), ProductRecommender (product recommendation system)

[1302] Data processing and calculation

[1303] 1. User Input: The user inputs room images and requirements using a chat-based interface on their smartphone, and also captures real-world room data using the smartphone camera.

[1304] 2. Generative AI: The server uses a generative AI model to generate the optimal layout based on the input data. This generative AI model has learned various room layouts.

[1305] 3. Emotion recognition: Using the user's facial image captured by the smartphone camera, EmotionRecognizer analyzes the user's emotions in real time.

[1306] 4. Product Recommendation: Based on the emotional data obtained by the emotion recognition method, ProductRecommender recommends the most suitable product. This recommendation is based on the results of learning the user's purchasing history and emotional data.

[1307] 5. Output method: The generated layout and information on recommended products (manufacturer name, product name, price, etc.) are displayed on the user's smartphone.

[1308] Specific examples

[1309] When a user uses their smartphone in a store to scan the shelves with their camera, their emotions are analyzed in real time and products that they might be interested in are recommended. For example, if the user smiles, related products (e.g., snacks or drinks) are recommended based on that emotion.

[1310] Prompt Sentence Examples

[1311] "Develop an application that analyzes emotions in real time when a user scans a shelf with their smartphone camera and recommends products based on their emotions. Use the EmotionRecognizer model for emotion recognition and the ProductRecommender system for product recommendations."

[1312] In this way, it becomes possible to propose optimal layouts and recommend products that take into account the user's emotions and purchasing history, thereby increasing user satisfaction.In addition, by proposing product and shelf placement based on behavioral economics, it is possible to achieve optimal product placement in commercial facilities.

[1313] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1314] Step 1:

[1315] A user uses a chat-based interface on their smartphone to input their image of the room and their requirements, and the text data is sent to a server, which receives the data and passes it to a generative AI model.

[1316] Step 2:

[1317] The user captures real-world data of the room using the smartphone camera. The captured image data is sent to the server, which receives the data and stores it as real-world data.

[1318] Step 3:

[1319] The server uses a generative AI model to generate an optimal layout based on the input text data and real-world data, and the generated layout data is stored within the server.

[1320] Step 4:

[1321] The user captures a facial image using the smartphone camera. The captured facial image is then sent to the server, which receives the data and passes it to the emotion recognition unit.

[1322] Step 5:

[1323] The server uses the emotion recognition means to analyze the user's emotions from the captured facial image in real time, and the emotion data obtained as a result of the analysis is stored in the server.

[1324] Step 6:

[1325] The server uses the product recommendation means to recommend optimal products based on the emotion data obtained by the emotion recognition means, and the recommended product data is stored in the server.

[1326] Step 7:

[1327] The server then sends the generated layout data and recommended product data to the user's smartphone, which receives and displays the data on its screen.

[1328] Step 8:

[1329] When a user selects a recommended product, the selection data is sent to the server, which receives the data and stores it as the user's purchasing history.

[1330] Step 9:

[1331] The server learns the user's purchasing history and emotional data and uses learning methods to optimize the next recommendation. The learning results are stored in the server and reflected in the next recommendation.

[1332] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1333] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1334] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1335] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1336] [Third embodiment]

[1337] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1338] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1339] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1340] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1341] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1342] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1343] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1344] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1345] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1346] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1347] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1348] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1349] "Example 1"

[1350] In one example of the present invention, a user inputs an image of their room (e.g., "a living room with a simple and calm atmosphere") through a chat-based interface. This input is analyzed by the generative AI and combined with previously learned room layout data to generate new layout proposals.

[1351] "Example 2"

[1352] In addition, users input real-world environmental data (e.g., data captured by LiDAR, images, videos, etc.) into the system. This data is analyzed by generative AI, and a layout proposal tailored to the real environment is generated.

[1353] "Example 3"

[1354] The generated layout proposal includes specific product information (e.g., manufacturer name, product name, price, etc.) for the furniture and home appliances to be placed, allowing users to use this information as a reference for specific product selection.

[1355] "Example 4"

[1356] Furthermore, an example embodiment of the present invention includes an extension function that makes optimal suggestions based on behavioral economics regarding the placement of products and product shelves in commercial facilities, etc. This function makes it possible to propose optimal product placement that encourages customer purchasing behavior.

[1357] The processing flow of each embodiment will be described below.

[1358] "Example 1"

[1359] Step 1: The user inputs an image of their room (e.g., "a living room with a simple and calm atmosphere") through a chat-based interface.

[1360] Step 2: This input is parsed by the generative AI.

[1361] Step 3: The generation AI combines the learned room layout data with input from the user to generate new layout proposals.

[1362] "Example 2"

[1363] Step 1: The user inputs real-world environmental data (e.g., data captured by LiDAR, images, videos, etc.) of the target room into the system.

[1364] Step 2: This data is analyzed by the generative AI.

[1365] Step 3: The generation AI combines the real-world data with the learned room layout data to generate layout proposals tailored to the real-world environment.

[1366] "Example 3"

[1367] Step 1: The generated layout proposal includes specific product information (e.g., manufacturer name, product name, price, etc.) of the furniture and home appliances to be placed.

[1368] Step 2: The user uses this product information to make a specific product selection.

[1369] "Example 4"

[1370] Step 1: The system will include an extension that makes optimal suggestions based on behavioral economics regarding product and shelf placement in commercial facilities, etc.

[1371] Step 2: This feature suggests optimal product placement to encourage customer purchasing behavior.

[1372] Example 1

[1373] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1374] Conventional room layout suggestion systems have difficulty proposing layouts that reflect the user's specific image, and making suggestions that are appropriate for the actual environment. Furthermore, they lack specific furniture and appliance suggestions based on the user's desired atmosphere and style.

[1375] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1376] In this invention, the server includes a chat-based image input means for a user, a means for transmitting the input image to the server, a means for the server to pass the received image to a generative AI model, a means for the generative AI model to analyze the input and combine it with learned layout data to generate a new layout proposal, a means for the server to transmit the generated layout proposal to a terminal, and a means for the terminal to display the layout proposal to the user. This makes it possible to propose a layout based on the user's specific image, and also to propose specific furniture and home appliances suitable for the actual environment.

[1377] The "chat-based image input means from the user" is a means for providing an interface for the user to input an image of the room in text format.

[1378] The "means for transmitting the input image to the server" is a communication means for transmitting the text data input by the user to the server.

[1379] The "means for passing the image received by the server to the generative AI model" is a processing means for passing the text data received by the server to the generative AI model.

[1380] "The means by which the generative AI model analyzes input and combines it with previously learned layout data to generate new layout proposals" refers to the means by which the generative AI model analyzes user input and combines it with previously learned data to generate new layout proposals.

[1381] The "means by which the server transmits the generated layout proposal to the terminal" refers to a communication means by which the server transmits the layout proposal received from the generative AI model to the terminal.

[1382] The "means for the terminal to display the layout proposal to the user" refers to a display means for the terminal to display the layout proposal received from the server to the user.

[1383] This invention is a system in which a user inputs an image of a room through a chat-based interface, and a generative AI model generates new layout proposals based on that image. Specific embodiments of this system are described below.

[1384] Users access the chat interface using a smartphone or PC browser, which allows them to input a text image of the room they want to live in. For example, they might enter "a living room with a simple and relaxing atmosphere."

[1385] The device sends the text data entered by the user to the server as an HTTP POST request. This request includes the user's input text. The server receives the HTTP POST request, extracts the user's input text from the request body, and then creates an API request to pass this text to the generative AI model and sends it to the generative AI model.

[1386] The generative AI model uses a model that uses natural language processing technology (e.g., GPT-4). This model analyzes the user's input text and combines it with previously learned room layout data to generate new layout proposals. Based on the analysis results, it can generate layout proposals such as the following:

[1387] The wall color is light gray

[1388] The sofa has a simple design and is beige in color.

[1389] The coffee table is made of wood and has a natural color.

[1390] The lighting makes extensive use of indirect lighting to provide soft light.

[1391] The server sends the layout proposal received from the generative AI model to the device as an HTTP response. This response contains the generated layout proposal. The device parses the HTTP response received from the server and extracts the layout proposal. The device then displays the proposal on the chat interface. The user can review the proposed layout and re-enter it if necessary.

[1392] For example, if a user inputs "I want to create a simple and calm living room," a generative AI model might generate the following layout suggestions:

[1393] The wall color is light gray

[1394] The sofa has a simple design and is beige.

[1395] The coffee table is made of wood and has a natural color.

[1396] The lighting makes extensive use of indirect lighting to provide soft light.

[1397] Example prompt sentence:

[1398] User: I want to create a simple and calm living room.

[1399] Server: Analyzing. Please wait a moment.

[1400] Generative AI model: The wall color is pale gray, the sofa is simple and beige, the coffee table is made of wood and has a natural color, and the lighting is indirect, providing soft light.

[1401] Server: The proposed layout is as follows:

[1402] Terminal: The wall color is pale gray, the sofa is simply beige, the coffee table is made of wood and has a natural color, and the lighting is indirect, providing soft light.

[1403] In this way, users can easily receive room layout suggestions through a chat-based interface.

[1404] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1405] Step 1:

[1406] The user inputs an image of the room into the chat interface.

[1407] The user opens a browser on their smartphone or PC, accesses the chat interface, types in, for example, "a living room with a simple and calm atmosphere," and clicks the send button. The input data is in text format.

[1408] Step 2:

[1409] The device sends input to the server.

[1410] The terminal sends the text data entered by the user to the server as an HTTP POST request. This request contains the user's input text. The input data is in text format, and the output data is an HTTP request.

[1411] Step 3:

[1412] The server receives the input and passes it to the generative AI model.

[1413] The server receives the HTTP POST request, extracts the user's input text from the request body, creates an API request to pass this text to the generative AI model, and sends it to the model. The input data is in text format, and the output data is an API request.

[1414] Step 4:

[1415] The generative AI model analyzes the input and combines it with learned layout data to generate new layout proposals.

[1416] The generative AI model receives API requests and analyzes the user's input text. Based on the analysis results, it combines them with the learned room layout data to generate new layout proposals. For example, it generates proposals such as "The wall color should be light gray, the sofa should have a simple beige design, the coffee table should be made of wood and have a natural color, and the lighting should be indirect, providing soft light." The input data is in text format, and the output data is the layout proposal.

[1417] Step 5:

[1418] The server receives the generated layout proposal and transmits it to the terminal.

[1419] The server sends the layout proposal received from the generative AI model to the terminal as an HTTP response. This response contains the generated layout proposal. The input data is the layout proposal, and the output data is the HTTP response.

[1420] Step 6:

[1421] The terminal displays the layout proposal to the user.

[1422] The device analyzes the HTTP response received from the server and extracts layout suggestions. It then displays these suggestions on the chat interface. The user can review the proposed layout and re-enter it if necessary. The input data is the HTTP response, and the output data is the displayed layout suggestions.

[1423] (Application example 1)

[1424] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1425] Conventional interior design support systems have difficulty generating layout proposals that specifically reflect the user's desired image of the space. Furthermore, there is a lack of tools to efficiently support the interior design of brick-and-mortar stores, and there has been a demand for a system that allows users to easily receive layout proposals using their smartphones.

[1426] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1427] In this invention, the server includes a generative AI means that has learned the layouts of various spaces, a chat-based image input means from a user, a means for inputting real-world environmental data of the target space, a means for generating / proposing a layout that suits the real environment, a means for outputting product information of items to be placed, a means for supporting the interior design of a physical store, a means including an application to be installed on a smartphone, and a means for generating a layout proposal using the generative AI model. This makes it possible to generate a layout proposal that specifically reflects the image of the space desired by the user, and to efficiently support the interior design of a physical store.

[1428] "Generative AI means" is an artificial intelligence technology that learns the layouts of various spaces and generates new layout proposals based on user input.

[1429] The "chat-based image input means" is an interface that allows users to input an image of the desired space in text format.

[1430] "Means for inputting real-world data" refers to a means for acquiring the current state of the target space as data and inputting it into the system.

[1431] "Means for generating / proposing layouts" refers to means for proposing new layouts to users based on data analyzed by the generation AI means.

[1432] The "means for outputting product information of an item" is a means for providing the user with detailed information about the items to be arranged in the layout.

[1433] "Means to support the interior design of physical stores" refers to means that provide support for efficiently designing the space of physical stores.

[1434] "Means including an application installed on a smartphone" means means that allows a user to use the system through an application that runs on a smartphone.

[1435] "Means for generating layout proposals using a generative AI model" means means for generating layout proposals based on user input using a generative AI model.

[1436] A system for implementing this invention is configured as follows: First, a user uses an application installed on a smartphone to input an image of a desired space into a chat-based interface. This input is analyzed by a generative AI means. The generative AI means has learned the layouts of various spaces and generates new layout proposals based on the user's input.

[1437] The server receives image input from the user and acquires current spatial data using a means of inputting real-world environmental data for the target space. This data can be collected, for example, using a smartphone camera or sensors. The server then uses generative AI means to generate and propose a layout that fits the real-world environment.

[1438] The generated layout proposal is displayed to the user through a smartphone application, and product information for the items to be placed is also displayed at the same time. This allows the user to purchase the necessary items based on the proposed layout.

[1439] As a concrete example, if the user requests a "cafe with a simple and relaxing atmosphere," the following prompt sentence is generated:

[1440] Example prompt sentence:

[1441] The image of the store desired by users: A simple and calm cafe

[1442] Generate a new layout proposal.

[1443] By inputting this prompt into a generative AI model, new store layout proposals that match the user's preferences can be obtained. For example, OpenAI's GPT-3 model can be used as a generative AI model. This model analyzes the user's input and generates optimal layout proposals based on the data it has already learned.

[1444] This system allows users to easily receive interior design suggestions using their smartphones, making it possible to efficiently design the interiors of physical stores.

[1445] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1446] Step 1:

[1447] Users launch an application installed on their smartphone and input an image of the space they want into a chat-based interface. The input data is in text format and includes a specific image, such as "a cafe with a simple and relaxing atmosphere." This input data is then passed on to the next step of processing.

[1448] Step 2:

[1449] The terminal receives image input from the user and sends it to the generation AI means. The generation AI means analyzes the input text data and generates a prompt sentence. This prompt sentence is converted into a format that can be input to the generation AI model. For example, the prompt sentence might be, "The image of the store desired by the user: A cafe with a simple and relaxing atmosphere. Please generate a new layout proposal."

[1450] Step 3:

[1451] The server inputs the generated prompt sentence into a generative AI model (e.g., OpenAI's GPT-3 model), which analyzes the prompt sentence and generates new layout proposals based on the learned data. These layout proposals are then passed to the next step.

[1452] Step 4:

[1453] The server receives the generated layout proposal and combines it with real-world data of the target space. The real-world data is acquired using the smartphone's camera and sensors. This data includes the space's dimensions and the layout of existing furniture. The server integrates this data and generates an optimal layout proposal.

[1454] Step 5:

[1455] The server sends the final layout proposal to the smartphone application. The device displays this proposal to the user. The user can review the proposed layout and make any necessary modifications. Product information for the items to be placed is also displayed at the same time, allowing the user to purchase the items they need.

[1456] Step 6:

[1457] If the user is satisfied with the proposed layout, they finalize it. The device then sends this information to the server, which then performs subsequent processes such as saving the layout and ordering items. This allows the user to efficiently realize their interior design.

[1458] Example 2

[1459] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1460] Conventional room layout proposal systems are limited in the types and formats of data that users can input, making it difficult to provide specific proposals that are suited to the actual environment. Furthermore, the generated layout proposals often do not meet the user's expectations, resulting in the need to make re-proposals, which is time-consuming. Furthermore, it is difficult to confirm whether the proposed layout is actually possible.

[1461] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1462] In this invention, the server includes a means for preprocessing received data, a means for inputting the preprocessed data into a generative AI model, and a means for returning the generated layout proposal to the user. This makes it possible to accommodate a variety of data formats input by users and to provide specific layout proposals that are suited to the actual environment. Furthermore, even if the generated layout proposal does not meet the user's expectations, a new proposal can be quickly made. Furthermore, it becomes easy to confirm whether the proposed layout can actually be arranged.

[1463] "Generative AI means" refers to artificial intelligence technology that learns various room layouts and generates optimal layout suggestions based on input data.

[1464] "Chat-based image input means" refers to an interface that allows users to input images and requests in text format.

[1465] "Means for inputting real-world data" refers to an interface that allows users to upload real-world data, such as LiDAR data, images, and videos of the target room, to the system.

[1466] "Means for generating / proposing layouts" refers to the function in which the AI ​​generating means generates the optimal room layout based on the input real-world environment data and proposes it to the user.

[1467] "Means for outputting product information" refers to a function that provides the user with information about the furniture and home appliances to be placed based on the generated layout.

[1468] "Means for preprocessing received data" refers to the process of analyzing data such as LiDAR data, images, and videos received from users and extracting necessary information.

[1469] "Means for inputting preprocessed data into a generative AI model" refers to the process of inputting preprocessed data into a generative AI means to generate optimal layout proposals.

[1470] "Means for returning the generated layout proposals to the user" refers to an interface for providing the layout proposals generated by the generating AI means to the user.

[1471] This invention is a system that allows a user to input real-world environment data of a target room and generates an optimal layout proposal using a generative AI model. Specific embodiments of this system are described below.

[1472] First, the user inputs real-world data from the target room into the system. This data includes 3D data acquired using a LiDAR scanner, images and videos taken with a smartphone, etc. This data is uploaded from the user's device through the system interface.

[1473] The server receives data uploaded by users. The received data may be in the form of LiDAR data, images, videos, etc. The server first preprocesses this data. Specifically, it converts the LiDAR data into a 3D model and extracts room features from the images and videos. For example, it extracts information such as wall color, furniture placement, and window positions.

[1474] The server then inputs the preprocessed data into a generative AI model, which uses advanced AI techniques such as OpenAI's GPT-4 or DALL-E, to generate an optimal layout based on the input data.

[1475] The server generates specific layout proposals based on the analysis results obtained from the generative AI model. These proposals include furniture placement, design ideas, color combinations, and more. The generated layout proposals are returned to the user. The user can review the proposals through the system interface and request revisions or re-proposals as needed.

[1476] As a concrete example, consider a user who wants to improve the layout of their living room. The user uses a LiDAR scanner to obtain 3D data of the living room and takes photos and videos of the room with their smartphone. These data are then uploaded to the system. The server first analyzes the LiDAR data and generates a 3D model of the room. It then extracts information such as furniture placement and wall color from the images and videos. Based on this data, a generative AI model generates optimal layout suggestions.

[1477] An example of a specific prompt is, "Please analyze the 3D data and images of the living room and suggest optimal furniture placement and design ideas." In this way, users can receive specific layout suggestions from the generative AI model based on the real-world environment data of the room.

[1478] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1479] Step 1:

[1480] The user inputs real-world data

[1481] The user inputs real-world data of the target room into the system. Specifically, the user acquires 3D data of the room using a LiDAR scanner, takes photos and videos of the room using a smartphone, and uploads this data through the system interface. The input data includes LiDAR data, images, and videos.

[1482] Step 2:

[1483] The server receives the data

[1484] The server receives the data uploaded by the user, which may be in the form of LiDAR data, images, videos, etc. The server temporarily stores this data and prepares it for the next processing step.

[1485] Step 3:

[1486] The server preprocesses the data

[1487] The server preprocesses the received data. Specifically, it converts the LiDAR data into a 3D model and extracts room features from images and videos. For example, it extracts information such as wall color, furniture placement, and window position. The input data are LiDAR data, images, and videos, and the output data are the 3D model and room feature information.

[1488] Step 4:

[1489] The server analyzes the data using a generative AI model

[1490] The server inputs the preprocessed data into a generative AI model, which uses advanced AI technology, for example. The generative AI model generates an optimal layout based on the input data. The input data is a 3D model and room feature information, and the output data is a layout proposal.

[1491] Step 5:

[1492] The server generates a layout proposal

[1493] The server generates specific layout proposals based on the analysis results obtained from the generative AI model. These proposals include furniture placement, design ideas, color combinations, etc. The input data is the analysis results of the generative AI model, and the output data is the specific layout proposals.

[1494] Step 6:

[1495] The server returns the proposal to the user

[1496] The server returns the generated layout proposal to the user. The user can review the proposal through the system interface and request corrections or resubmissions as necessary. The input data is the specific layout proposal, and the output data is the proposal information to the user.

[1497] (Application example 2)

[1498] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1499] Conventional store layout optimization relies heavily on experience and intuition, making it difficult to achieve efficient layouts. In addition, there has been a lack of layout proposals that utilize real-world data, making it difficult to achieve optimal layouts that are in line with the actual store environment.

[1500] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout that suits the real environment, a means for outputting product information of the furniture / appliances to be placed, a means for capturing real-world environment data of the store, a means for analyzing the data using a generation AI model and generating an optimal layout proposal, and a means for displaying the generated layout proposal. This enables efficient layout proposals that are suited to the actual store environment.

[1501] A "generative AI method" is an artificial intelligence method that learns various room layouts and generates the optimal layout based on input data.

[1502] A "chat-based image input means" is a means for users to input images and requests in text format.

[1503] "Means for inputting actual environment data" refers to means for inputting actual environment data of the target room or store into the system.

[1504] The "means for generating / proposing layouts" is a means for generating optimal layouts based on input real-world data and proposing them to the user.

[1505] The "means for outputting product information" is a means for providing detailed product information of the furniture and home appliances to be placed to the user.

[1506] "Means for capturing real-world store environment data" refers to means for obtaining real-world environment data such as LiDAR data, images, and videos within the store.

[1507] "Means for analyzing data using a generative AI model" refers to a means for inputting acquired real-world data into a generative AI model and performing analysis.

[1508] The "means for displaying a layout proposal" is a means for visually displaying the generated layout proposal to the user.

[1509] A system for implementing this invention is configured as follows. First, a user captures real-world data in a store using a smartphone. Specifically, the user acquires 3D data using a LiDAR sensor and takes images and videos using the smartphone's camera. This data is then sent to a server via an application on the smartphone.

[1510] The server uses a generative AI model to analyze the received real-world data. The generative AI model has previously learned about various room layouts and generates optimal layout proposals based on the input data. Specifically, the process is carried out as follows:

[1511] 1. Data capture:

[1512] Users use their smartphones to capture LiDAR data and image data inside the store.

[1513] This data is sent to a server via a smartphone application.

[1514] 2. Data Analysis:

[1515] The server inputs the received LiDAR data and image data into the generative AI model.

[1516] The generative AI model generates optimal layout suggestions based on the input data.

[1517] 3. Generate layout proposals:

[1518] The generative AI model analyzes real-world data and generates optimal layout proposals.

[1519] The generated layout proposal is displayed on the user's smartphone.

[1520] The hardware used includes a smartphone, LiDAR sensor, and smartphone camera, while the software used includes a generative AI model (e.g., GPT-3), an application for data capture, and a server for data analysis.

[1521] As a concrete example, the following prompt sentence is input to the generative AI model:

[1522] Example prompt sentence:

[1523] LiDAR data: [0.5, 1.2, 0.3, ...], Image data: [image_data_as_base64_string]

[1524] By inputting this prompt into a generative AI model, the model generates suggestions for optimizing the store layout, such as "rearrange product placement to improve customer flow" or "place specific products in prominent locations."

[1525] In this way, it becomes possible to propose an efficient layout that is suited to the actual store environment.

[1526] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1527] Step 1:

[1528] Users use their smartphones to capture real-world data in the store. Specifically, they acquire 3D data using a LiDAR sensor and take images and videos using the smartphone's camera. The input data is the LiDAR data and image data, which are then saved in the smartphone's application.

[1529] Step 2:

[1530] The user sends the captured real-world data to the server through a smartphone application. The input data is LiDAR data and image data, which are uploaded to the server via the network. The output is the real-world data stored on the server.

[1531] Step 3:

[1532] The server uses a generative AI model to analyze the received real-world data. Specifically, it inputs LiDAR data and image data into the generative AI model. The input data are LiDAR data and image data, and the generative AI model analyzes them to generate an optimal layout proposal. The output is a layout proposal.

[1533] Step 4:

[1534] The server sends the generated layout proposal to the user's smartphone. The input data is the layout proposal from the generative AI model, which is sent to the user's smartphone via the network. The output is the layout proposal displayed on the user's smartphone.

[1535] Step 5:

[1536] The user checks the generated layout proposal through a smartphone application. The input data is the layout proposal sent from the server, which is displayed on the smartphone screen. The output is the layout proposal that the user checks.

[1537] In this way, it becomes possible to propose an efficient layout that is suited to the actual store environment.

[1538] Example 3

[1539] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1540] Conventional layout proposal systems make it difficult for users to obtain specific product information and to determine whether the proposed layout is suitable for the actual environment. In addition, in commercial facilities, they are unable to propose optimal product placement to encourage customer purchasing behavior.

[1541] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout suited to the real environment, a means for outputting product information of the furniture / appliances to be arranged, a means for acquiring product information from a database based on the generated layout proposal, a means for incorporating the acquired product information into the layout proposal, and a means for returning the final layout proposal to the user. This allows the user to obtain a layout proposal including specific product information, making it easy to determine a layout suitable for the actual environment. Furthermore, in commercial facilities, optimal product placement can be proposed to promote customer purchasing behavior.

[1542] "Generative AI means" refers to artificial intelligence that learns various room layouts and generates optimal layout suggestions based on the user's requirements.

[1543] "Chat-based image input means" refers to an interface that allows users to input layout preferences and conditions in text format.

[1544] "Means for inputting real-world data" refers to a means for inputting real-world environmental data, such as the dimensions and shape of the target room and the existing furniture layout, into the system.

[1545] "Means for generating / proposing layouts" refers to means for generating an optimal room layout based on input data and proposing it to the user.

[1546] "Means for outputting product information" refers to means for providing the user with specific product information (manufacturer name, product name, price, etc.) of the furniture and home appliances included in the generated layout.

[1547] The "means for acquiring product information from a database" refers to a means for searching and acquiring necessary product information from a database based on the generated layout proposal.

[1548] The "means for incorporating product information into a layout proposal" refers to a means for integrating acquired product information into a generated layout proposal and providing it to a user.

[1549] The "means for returning a final layout proposal to the user" refers to a means for transmitting the final generated layout proposal to the user.

[1550] This invention is a system for allowing a user to receive room layout proposals, and generates layout proposals including specific product information using a generative AI model. A specific embodiment of this system is described below.

[1551] Server Roles

[1552] The server receives requests from users and generates layout proposals using generative AI models. The server uses the following hardware and software:

[1553] Hardware: High-performance server machine (e.g., CPU, memory, storage)

[1554] Software: Generative AI model, database management system, API server

[1555] The server receives a request from the user, generates and inputs a prompt to the generative AI model, analyzes the generated layout proposal, and retrieves the necessary product information from the database. The retrieved product information is then incorporated into the layout proposal and sent back to the user.

[1556] Device Role

[1557] The device sends user-entered requests to the server and displays layout suggestions returned by the server. The device uses the following hardware and software:

[1558] Hardware: PCs, smartphones, tablets

[1559] Software: Web browser, dedicated application

[1560] The terminal provides an interface for the user to input requests, transmits the requests to the server, receives layout proposals returned from the server, and displays them to the user.

[1561] User Roles

[1562] Users can request a layout proposal through their device, review the returned proposal, and use the specific product information included in the proposal to help them select furniture and home appliances.

[1563] Specific examples

[1564] Here are some example prompts to input to the generative AI model:

[1565] "Please suggest a living room layout that meets the following criteria:

[1566] Includes sofa, TV and coffee table

[1567] Modern design

[1568] The budget is under 200,000 yen.

[1569] When this prompt is sent to the server, the generative AI model generates a living room layout proposal and returns it, including specific product information (e.g., manufacturer name, product name, price, etc.). The user can use this information to select furniture and appliances.

[1570] In this way, the user can obtain layout proposals including specific product information, and can easily determine a layout suitable for the actual environment. Also, in commercial facilities, the system can propose optimal product placement to promote customer purchasing behavior. The flow of the specification process in the third embodiment will be described with reference to FIG. 15.

[1571] Step 1: Receiving a user request

[1572] The server receives a layout proposal request from the user. Specifically, it receives an HTTP request and analyzes the request content. The input is the request data sent by the user from the terminal, and the output is the analyzed request content.

[1573] Step 2: Generate a prompt statement

[1574] The server generates a prompt sentence to be input to the generative AI model based on the user's request. Specifically, it uses a template to construct a prompt sentence based on the user's input data. The input is the parsed request content, and the output is the generated prompt sentence.

[1575] Step 3: Input to the generative AI model

[1576] The server inputs the generated prompt sentence into the generative AI model and generates a layout proposal. Specifically, it calls the API of the generative AI model and sends the prompt sentence. The input is the generated prompt sentence, and the output is the layout proposal returned from the generative AI model.

[1577] Step 4: Get product information

[1578] The server retrieves the necessary product information from a database based on the generated layout proposal. Specifically, it executes a database query to retrieve information such as the manufacturer, product name, and price of the furniture or home appliance. The input is the generated layout proposal, and the output is the retrieved product information.

[1579] Step 5: Assembling the layout proposal

[1580] The server incorporates the acquired product information into the layout proposal to create the final proposal. Specifically, it integrates the generated layout proposal with the product information and constructs data to be sent back to the user. The input is the acquired product information, and the output is the final layout proposal.

[1581] Step 6: Return to user

[1582] The server returns the final layout proposal to the user. Specifically, it generates an HTTP response and sends it to the user's device. The input is the final layout proposal, and the output is the layout proposal sent to the user's device.

[1583] Step 7: Providing a Request Interface

[1584] The terminal provides an interface for the user to request a layout proposal. Specifically, it displays a form and input fields to allow the user to enter conditions. The input is the user's operation, and the output is the request data entered by the user.

[1585] Step 8: Submitting the Request

[1586] The terminal sends the request entered by the user to the server. Specifically, when the user submits a form, an HTTP request is sent to the server. The input is the request data entered by the user, and the output is the request sent to the server.

[1587] Step 9: Receiving the Response

[1588] The terminal receives the layout proposal returned from the server. Specifically, it receives the HTTP response and analyzes the data. The input is the layout proposal returned from the server, and the output is the analyzed layout proposal.

[1589] Step 10: View layout suggestions

[1590] The terminal displays the received layout proposal to the user. Specifically, the layout proposal and product information are displayed on the screen so that the user can confirm them. The input is the analyzed layout proposal, and the output is the layout proposal displayed to the user.

[1591] Step 11: Enter your request

[1592] The user requests a layout proposal through the terminal interface. Specifically, the user enters the conditions into a form and clicks the submit button. The input is the user's desired conditions, and the output is the request data entered into the terminal.

[1593] Step 12: Review the proposal

[1594] The user checks the layout proposal returned from the server. Specifically, the user views the layout proposal and product information displayed on the terminal. The input is the layout proposal displayed on the terminal, and the output is the user's confirmation result.

[1595] Step 13: Product Selection Recommendations

[1596] The user selects furniture or home appliances based on the specific product information included in the proposal. Specifically, the user decides which furniture or home appliance to purchase based on the proposed product information. The input is the proposed product information, and the output is the user's selection.

[1597] (Application example 3)

[1598] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1599] Product placement in traditional stores often relied on experience and intuition, making it difficult to find the optimal placement. Furthermore, there was a lack of means to provide specific product information, leaving users with a lack of reference information to use when making product selections. Furthermore, it was not possible to incorporate a scientific approach based on behavioral economics, making it difficult to effectively promote customer purchasing behavior.

[1600] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1601] In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from the user, a means for inputting actual environment data of the target room, a means for generating / proposing a layout that suits the actual environment, a means for outputting product information for the furniture / appliances to be arranged, a means for proposing product arrangements based on behavioral economics, a means for providing a smartphone application for optimizing product arrangements in a store, and a means for providing specific product information (manufacturer name, product name, price, etc.).By scientifically optimizing product arrangements in a store and providing specific product information, it is possible to enrich reference information for users when making product selections and effectively promote customer purchasing behavior.

[1602] A "generative AI means" is an artificial intelligence means that learns various room layouts and generates the optimal layout based on the user's requirements.

[1603] A "chat-based image input means" is a means for users to input images and requests in a chat format.

[1604] The "means for inputting actual environment data" refers to a means for inputting actual environment data of the target room or store.

[1605] The "means for generating / proposing a layout" is a means for generating and proposing an optimal layout based on input data.

[1606] The "means for outputting furniture / home appliance product information" is a means for outputting specific product information (manufacturer name, product name, price, etc.) of the furniture or home appliance to be arranged.

[1607] "Means for proposing product placement based on behavioral economics" refers to means for proposing optimal product placement that promotes customer purchasing behavior based on the theory of behavioral economics.

[1608] The "means for providing a smartphone application" is a means for providing an application on a smartphone for optimizing product placement in a store.

[1609] The "means for providing specific product information" refers to a means for providing specific product information such as the manufacturer name, product name, price, etc. to the user.

[1610] To implement the present invention, the following system configuration and processing procedures are required.

[1611] First, the server is equipped with a generation AI means that has learned various room layouts. This generation AI means is an artificial intelligence that generates the optimal layout based on past layout data. Users can input their desired layout and image using a chat-based image input means. This communicates the user's specific requests to the system.

[1612] Next, the server is equipped with a means for inputting the actual environment data of the target room or store. This means inputs data such as the dimensions and layout of the actual room or store. Based on this data, the generation AI means generates and proposes a layout that suits the actual environment.

[1613] Furthermore, the server is equipped with a means for outputting product information for the furniture and home appliances to be placed. This means allows the user to check specific product information (manufacturer name, product name, price, etc.), which can be used as a reference for the user in selecting specific products.

[1614] The server also has a means for proposing product placement based on behavioral economics. This allows it to propose optimal product placement that encourages customer purchasing behavior, thereby scientifically optimizing product placement within the store.

[1615] Furthermore, the server includes a means for providing a smartphone application for optimizing product placement in a store. By using this application, the store manager or staff can optimize product placement using their smartphones.

[1616] The server also has a means for providing specific product information, which allows the user to obtain specific product information such as the manufacturer's name, product name, price, etc.

[1617] The hardware used is a smartphone, and the software uses Python and the OpenAI API. Data processing involves formatting store data (product list, current layout information) in JSON format, and then using a generative AI model to generate the optimal product placement.

[1618] As a concrete example, by inputting the following prompt sentence into the generative AI model, optimal product placement suggestions can be obtained.

[1619] Example prompt sentence:

[1620] Store data: {"Store name": "ABC Store", "Product list": [{"Manufacturer name": "Manufacturer A", "Product name": "Product A", "Price": 1000}, {"Manufacturer name": "Manufacturer B", "Product name": "Product B", "Price": 2000}], "Store layout": "Current layout information"}

[1621] Please suggest optimal product placement based on behavioral economics.

[1622] In this way, the server, terminal, and user can work together to achieve optimal product placement.

[1623] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1624] Step 1:

[1625] The user starts the application on their smartphone and inputs their desired layout and image using a chat-based image input means.

[1626] Input: The layout and image desired by the user

[1627] Output: User's desired layout and image data

[1628] Step 2:

[1629] The server receives the image data sent from the user and inputs data such as the dimensions and layout of the actual room or store using a means for inputting real environment data of the target room or store.

[1630] Input: User image data, real-world data

[1631] Output: Integrated layout data

[1632] Step 3:

[1633] The server uses a generation AI means to generate an optimal layout based on the integrated layout data. The generation AI means learns from past layout data and proposes an optimal layout taking into account the user's requests and actual environment data.

[1634] Input: Integrated layout data

[1635] Output: Optimal layout proposal

[1636] Step 4:

[1637] The server provides the user with specific product information (manufacturer name, product name, price, etc.) for the furniture and home appliances to be arranged based on the generated layout proposal, using a means for outputting product information.

[1638] Input: Optimal layout proposal

[1639] Output: Specific product information

[1640] Step 5:

[1641] The server uses a method based on behavioral economics to suggest optimal product placements that encourage customer purchasing behavior, thereby scientifically optimizing product placements within the store.

[1642] Input: Optimal layout proposals, behavioral economics data

[1643] Output: Optimal product placement proposal

[1644] Step 6:

[1645] The server notifies the store manager and staff of the optimal product placement using a means for providing a smartphone application for optimizing product placement within the store.

[1646] Input: Optimal product placement proposal

[1647] Output: Notification on smartphone application

[1648] Step 7:

[1649] Users can use the smartphone application to check specific product information and use it as a reference for product selection.

[1650] Input: Specific product information

[1651] Output: User selected product information

[1652] In this way, the server, terminal, and user can work together to achieve optimal product placement.

[1653] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1654] "Example 1"

[1655] One embodiment of the present invention provides a system incorporating an emotion engine. This system recognizes a user's emotions and adjusts room layout suggestions based on those emotions. Specifically, if a user inputs that they desire a "relaxed living room," the emotion engine recognizes the user's current emotional state (e.g., relaxed, tense, etc.). It then adjusts the layout suggestions to match the user's emotional state, for example, by suggesting furniture with more subdued colors if the user is relaxed, or by suggesting furniture arrangements that create a relaxing effect if the user is tense.

[1656] "Example 2"

[1657] The emotion engine is also used to suggest product choices. For example, if a user inputs "I'm looking for a new TV," the emotion engine will recognize the user's current emotional state and suggest product choices based on that emotional state. For example, if the user is feeling happy, it will suggest a large-screen, high-end TV, and if the user is feeling sad, it will suggest a cost-effective TV.

[1658] "Example 3"

[1659] Furthermore, the emotion engine learns changes in the user's emotions and optimizes layout suggestions and product selection suggestions based on the results of that learning. For example, if a user feels a strong sense of joy when looking at a certain product, the engine will prioritize suggestions for that product or products in the same category. The engine learns how users respond to emotions and makes suggestions that reflect this.

[1660] "Example 4"

[1661] The emotion engine also analyzes users' emotional responses and makes product placement suggestions based on the results. For example, if a user expresses joy at the placement of a certain product shelf, the engine will apply that same placement to other product shelves.

[1662] The processing flow of each embodiment will be described below.

[1663] "Example 1"

[1664] Step 1: The user inputs their desired room layout into the system.

[1665] Step 2: The emotion engine recognizes the user's current emotional state.

[1666] Step 3: Adjust the layout proposal based on the emotional state recognized by the emotion engine.

[1667] "Example 2"

[1668] Step 1: The user enters their product selection preferences into the system.

[1669] Step 2: The emotion engine recognizes the user's current emotional state.

[1670] Step 3: Tailor product selection suggestions based on the emotional state recognized by the emotion engine.

[1671] "Example 3"

[1672] Step 1: The emotion engine learns the user's emotional changes.

[1673] Step 2: Based on the results learned by the emotion engine, layout suggestions and product selection suggestions are optimized.

[1674] "Example 4"

[1675] Step 1: The emotion engine analyzes the user's emotional response.

[1676] Step 2: Based on the results of the emotion engine's analysis, product placement suggestions are made.

[1677] Example 1

[1678] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1679] Conventional room layout suggestion systems make uniform suggestions without considering the user's emotional state, making it difficult to provide a layout that suits the user's current psychological state. In addition, layouts generated based on user input may not match the actual environment or emotional state, resulting in a decrease in user satisfaction.

[1680] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a generation AI means that has learned various room layouts, a chat-based image input means from a user, a means for inputting real-world environment data of the target room, a means for generating / proposing a layout suited to the real environment, a means for outputting product information of furniture / appliances to be placed, an emotion engine means for recognizing the emotional state of the user, and a means for adjusting the layout proposal based on the emotional state. This makes it possible to propose a layout suited to the emotional state of the user, thereby improving user satisfaction.

[1681] "Generative AI means" is an artificial intelligence technology that learns various room layouts and generates new layout suggestions based on user input.

[1682] The "chat-based image input means" is an interface that allows users to input their image and preferences for the room in text format.

[1683] The "means for inputting real environment data" refers to a means for inputting environmental data such as the actual dimensions and layout of the target room into the system.

[1684] "Means for generating / proposing layouts" refers to means for proposing layouts generated by the generation AI means to users.

[1685] The "means for outputting furniture / home appliance product information" is a means for providing the user with detailed information about the furniture and home appliances that will be arranged based on the proposed layout.

[1686] "Emotion engine means" is a technology for recognizing the current emotional state of a user.

[1687] The "means for adjusting layout proposals based on emotional state" refers to means for optimizing the generated layout proposals in response to the perceived emotional state of the user.

[1688] This invention is a system in which a user inputs a room image through a chat-based interface, and a generative AI model makes layout suggestions based on that input. Furthermore, by combining it with an emotion engine, it provides optimal layout suggestions according to the user's emotional state.

[1689] The server uses the following hardware and software to realize this system: For the generative AI model, it uses OpenAI's GPT-4, for example. For the emotion engine, it uses Affectiva's emotion recognition API, for example. By combining these technologies, it generates layout proposals based on the user's input and emotional state.

[1690] Using a chat-based interface (e.g., a web application or mobile app), a user inputs an image of the room they want, such as "a simple, calm living room." This input is sent to a generative AI model running on a server.

[1691] The generative AI model analyzes user input and combines it with previously learned room layout data to generate initial layout suggestions. For example, for a "simple and calm living room," it suggests wall colors, furniture placement, lighting types, etc.

[1692] The server then uses an emotion engine to recognize the user's current emotional state: if the user provides emotion data through the camera or microphone, the emotion engine analyzes that data to determine whether the user is relaxed or tense.

[1693] The server adjusts the generative AI model's initial layout proposals based on the emotional state obtained from the emotion engine. For example, if the user is relaxed, it will suggest furniture with more muted colors, and if the user is tense, it will suggest furniture arrangements that have a relaxing effect.

[1694] As a concrete example, consider a case where a user inputs that they want a "simple and relaxing living room." If the emotion engine recognizes that the user is relaxed, the generative AI model will generate the following layout suggestions:

[1695] The wall color is light blue

[1696] The sofa is gray

[1697] The lighting is warm-colored indirect lighting.

[1698] On the other hand, if the emotion engine detects that the user is nervous, the generative AI model will generate layout suggestions like this:

[1699] The wall color is pale green

[1700] The sofa is beige

[1701] The lighting is soft white light

[1702] Example prompt sentence:

[1703] User input: "A simple, calm living room."

[1704] Emotional state: Relaxed

[1705] Output of the generative AI model: "The walls are pale blue, the sofa is gray, and the lighting is warm, indirect lighting."

[1706] In this way, the server can generate and provide optimal room layout suggestions to the user based on the user's input and emotional state.

[1707] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1708] Step 1:

[1709] The user inputs the image of the room through a chat-based interface. The user then enters specific details in text format, such as "a living room with a simple and calm...

Claims

1. A means for acquiring information about interior design from a user; A means for generating a prompt sentence to be input to the generative AI model based on the acquired information about the interior; a means for inputting the generated prompt sentence into the generative AI model and obtaining an interior layout proposal from the generative AI model; A means for outputting the acquired layout proposal; A system including:

2. The system according to claim 1 , further comprising: means for analyzing a user's emotions when obtaining interior information from the user using an emotion engine, and adjusting the content of the layout proposal based on the analyzed user's emotions.

3. The system according to claim 1 , further comprising: means for analyzing a user's emotion when the layout proposal is outputted using an emotion engine, and adjusting the content of the layout proposal based on the analyzed user's emotion.

Images