system

Abstract

Provide a system. A means for acquiring an image of a space by an imaging device; means for transmitting the image and related information to a server; a means for identifying spatial characteristics by an image analysis device in the server; means for generating optimal furniture and accessory recommendations using the analysis results and related information; means for transmitting the generated proposal to a terminal; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional interior coordination systems have had issues with the fact that users have to manually plan furniture placement and select interior items themselves, which is time-consuming. It was also difficult to automatically generate proposals that matched the user's budget and preferences, and detailed proposals tailored to individual needs were not provided. Furthermore, insufficient integration with interior design databases made it difficult to provide proposals that reflected the latest trends. [Means for solving the problem]

[0005] The present invention provides a system including a means for acquiring an image of a space using a photographing device, a means for transmitting the image and related information to a server, a means for identifying characteristics of the space using an image analysis device in the server, a means for providing a generator that generates optimal furniture and accessory proposals using the analysis results and related information, and a means for transmitting the generated proposals to a terminal.Furthermore, the present invention provides a means for acquiring information on budget, theme, and color, and a means for generating proposals in cooperation with a database of interior designs from around the world, thereby automatically proposing optimal interior coordination according to the individual needs of a user.

[0006] An "image capture device" is a device for capturing an image of a space.

[0007] A "server" is a computer system for receiving and processing images and related information.

[0008] An "image analysis device" is a device for extracting and analyzing spatial features from acquired images.

[0009] A "generator" is a device that generates optimal furniture and accessory recommendations based on the analysis results and related information.

[0010] A "terminal" is a device that allows a user to receive and check the proposed content.

[0011] "Budget" refers to the upper limit of costs set by the user for interior coordination.

[0012] "Theme" refers to the interior style or concept desired by the user.

[0013] "Color information" refers to information about the user's preferred colors and color coordination.

[0014] The "Interior Design Database" is a database that accumulates information and examples of interior design from around the world.

[0015] "Spatial features" refers to information such as the dimensions of the space, the layout of existing furniture, and the interior style analyzed from the captured images. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system in which a user acquires an image of a space, transmits the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. A specific embodiment of this system is described below.

[0038] User operation

[0039] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded via a dedicated application. This application provides the user with an interface for entering relevant information such as theme, color, and budget. The user enters this information and presses the "send" button to send the data to the server.

[0040] Server-side processing

[0041] The server receives the image and input information sent by the user. The server then sends the image data to an image analysis device, which analyzes the space's characteristics (e.g., dimensions, existing furniture layout, interior style). The analysis device extracts these characteristics and returns the analysis results to the server.

[0042] The server then sends these analysis results, along with information entered by the user such as theme, color, and budget, to the generator, which then generates optimal interior coordination proposals. The generator connects to interior design databases from around the world and can generate optimal item and layout proposals based on the latest trends.

[0043] Sending generated suggestions to users

[0044] The server then organizes the suggestions returned by the generators and sends them to the user's device, where they can be viewed within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[0045] Specific examples

[0046] For example, if User A has a budget of 100,000 yen, a Nordic theme, and favorite colors of blue and white, the user takes a picture of their living room with their smartphone, enters the relevant information into a dedicated app, and submits it. The server uses an image analysis device to analyze the dimensions of User A's living room and the layout of existing furniture, and the generator then proposes optimal furniture and accessories based on the Nordic theme.

[0047] The proposals are sent to User A's smartphone, and User A can check the proposals within the app. If User A likes the proposed items, he or she can purchase them within the app. Through this series of processes, users can easily realize their ideal interior coordination.

[0048] The above is a specific description of the embodiment of the present invention. This system allows users to receive optimal interior design proposals that meet their individual needs without any effort on their part.

[0049] The processing flow will be explained below.

[0050] Step 1:

[0051] The user takes an image of the room using a camera (smartphone camera).

[0052] Step 2:

[0053] The user launches a dedicated application and uploads the captured images to the app.

[0054] Step 3:

[0055] The user inputs relevant information such as budget, theme, color preferences, etc. through the application interface.

[0056] Step 4:

[0057] The user presses the "send" button to send the image and input information to the server.

[0058] Step 5:

[0059] The server receives the image and input information sent by the user.

[0060] Step 6:

[0061] The server sends the image data to an image analysis device, which analyzes the characteristics of the space (dimensions, existing furniture layout, interior style).

[0062] Step 7:

[0063] The image analyzer analyzes the image and extracts spatial features, and the results are sent back to the server.

[0064] Step 8:

[0065] The server transmits the analysis results and user input information to the generation device, which then generates an optimal interior proposal.

[0066] Step 9:

[0067] The generator generates optimal furniture and accessory proposals based on the analysis results and user input, linking with interior design databases around the world.

[0068] Step 10:

[0069] The server transmits the proposal content returned from the generating device to the user's terminal.

[0070] Step 11:

[0071] The user checks the proposal on their device (smartphone), which includes a furniture layout diagram, a list of recommended items, and a link to purchase.

[0072] Step 12:

[0073] The user selects the suggested item and checks out within the app.

[0074] This allows users to easily create their ideal interior coordination.

[0075] Example 1

[0076] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0077] Conventional interior coordination suggestion systems have the drawback of making it difficult for users to obtain proposals that match their preferences and budget, resulting in a time-consuming and labor-intensive process. Furthermore, obtaining optimal proposals that meet the user's needs requires advanced analysis and generation technology, but such technology is often not fully utilized.

[0078] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0079] In this invention, the server includes means for acquiring images of the space using a photographing device, means for transmitting the images and related information to the server, means for identifying the characteristics of the space using an image analyzing device in the server, means for providing a generating device for generating optimal furniture and accessory proposals using a generative AI model based on the analysis results and related information, and means for transmitting the generated proposals to a terminal. This enables a user to quickly and accurately receive interior coordination proposals based on the user's photographed images and desired conditions.

[0080] "Capture device" refers to a device for capturing images of a space. Examples include smartphones and digital cameras.

[0081] "Server" refers to a computer system that receives, stores, and processes data over a network.

[0082] "Image analysis device" refers to software or hardware for extracting and analyzing features from images. Specific examples include TENSORFLOW (registered trademark) and OpenCV.

[0083] "Generative AI model" refers to an artificial intelligence model that generates optimal suggestions based on user criteria. Specific examples include GPT-4 (registered trademark).

[0084] "Terminal" refers to a device that receives data from a server and is used as an interface by a user. Examples include smartphones and tablets.

[0085] "Spatial characteristics" refers to information such as the dimensions of the space, existing furniture layout, and interior style obtained through image analysis.

[0086] "Related information" refers to information entered by the user, such as theme, favorite color, budget, etc.

[0087] "Furniture and accessory suggestions" refers to specific interior layout ideas and lists of recommended items generated based on the user's criteria and analysis results.

[0088] "Database" refers to a system for storing and managing interior design data and other related information.

[0089] The present invention is a system in which a user acquires an image of a space, transmits the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. A specific embodiment of this system is described below.

[0090] User operation

[0091] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded via a dedicated application. This application provides the user with an interface for entering relevant information such as the theme, favorite colors, and budget. The user enters this information and presses the "send" button to send the data to the server.

[0092] Server-side processing

[0093] The server receives the image and related information sent by the user. The server then sends the image data to an image analyzer, requesting it to analyze the spatial features (e.g., dimensions, existing furniture layout, interior style). The image analyzer can use software such as TensorFlow or OpenCV. The analyzer extracts these features and returns the analysis results to the server.

[0094] The server then uses a generative AI model (such as GPT-4) to generate optimal interior coordination suggestions based on these analysis results and user information. The generator connects to interior design databases around the world to make suggestions based on the latest trends.

[0095] Sending generated suggestions to users

[0096] The server then organizes the suggestions returned by the generator and sends them to the user's device. The suggestions include a detailed furniture layout plan, a list of recommended items, and links to purchase them. The user can view these suggestions in a dedicated application.

[0097] Specific examples

[0098] For example, the specific operating procedure for User A, who has a budget of 100,000 yen, a Nordic theme, and preferred colors of blue and white, is shown below. User A first takes a picture of their living room with their smartphone and uploads it to a dedicated app. Next, they enter the relevant information into the app's interface and send it to the server. The server uses TensorFlow to analyze the dimensions and existing furniture layout of User A's living room, and uses GPT-4 to generate optimal interior design suggestions. The generated suggestions are sent to User A's smartphone, where User A can view the suggestions within the app.

[0099] Prompt Sentence Examples

[0100] In the case of User A, the generative AI model receives the following prompt: "Please suggest an interior design with a Nordic theme and blue and white colors for a budget of 100,000 yen."

[0101] In this way, by using the system of the present invention, the user can easily achieve the ideal interior coordination.

[0102] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0103] Processing Steps

[0104] Step 1:

[0105] The user takes a picture and sends it to the server through the application.

[0106] Input: User-taken images of your space, along with related information like theme, favorite colors, budget, etc.

[0107] How it works: The user takes a picture of their room using the smartphone camera, uploads it to a dedicated application, and then enters the theme, favorite colors, and budget information through the application interface and presses the send button to send it to the server.

[0108] Step 2:

[0109] A server receives the image and related information.

[0110] Output: Storage of image data and related information sent to the server

[0111] Specific operation: The server receives the image file, theme, favorite color, and budget information sent by the user, and stores them in a database in JSON format. The received data is linked to the user ID.

[0112] Step 3:

[0113] The server sends the image data to an image analysis device, which analyzes the characteristics of the space.

[0114] Input: Image data stored on the server

[0115] Output: Analysis results such as space dimensions, existing furniture arrangement, and interior style

[0116] How it works: The server sends the stored images to an image analyzer (e.g., TensorFlow or OpenCV), which extracts feature information from the images, such as the dimensions of the space, the existing furniture layout, and the interior style, and returns the analysis results to the server.

[0117] Step 4:

[0118] The server transmits the analysis results and user information to the generation device.

[0119] Input: Analysis results and related information entered by the user (theme, color, budget)

[0120] Output: Interior coordination suggestions generated by the generative AI model

[0121] Specific operation: The server combines the analysis results with the user's relevant information to create and send a prompt to the generative AI model (e.g., GPT-4). An example prompt is "Please suggest an interior coordination with a Nordic theme and blue and white colors for a budget of 100,000 yen." The generative AI model then generates the optimal interior coordination proposal and sends it back to the server.

[0122] Step 5:

[0123] The server organizes the generated results and sends them to the user terminal.

[0124] Input: Coordinate proposals from the generator

[0125] Output: Sending the organized proposals to the user's device

[0126] Specific operation: The server organizes the proposals received from the generator, formats them appropriately, and sends them to the user's device. The proposals include a detailed furniture layout plan, a list of recommended items, and links to purchase them.

[0127] Step 6:

[0128] The user reviews the suggestions within the application and purchases the items if necessary.

[0129] Input: Interior coordination proposal sent from the server

[0130] What it does: The user opens the app and sees the suggestions sent from the server, including a detailed furniture layout and a list of recommended items, and can purchase the items they like right within the app.

[0131] The above is the specific processing flow of the program for this system, as well as the input, output, and specific operation at each step.

[0132] (Application example 1)

[0133] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0134] Conventional interior coordination systems only allow users to check the proposed interior designs online, making it difficult for them to actually use and check the products. As a result, users are unable to check the actual appearance and texture of the products before purchasing them, which has the problem of not being able to fully stimulate users' purchasing motivation.

[0135] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0136] In this invention, the server includes means for capturing an image of the space using a photographing device, means for transmitting the image and related information to the server, means for identifying characteristics of the space using an image analyzing device in the server, means for providing a generating device for generating optimal furniture and accessory proposals using the analysis results and related information, means for transmitting the generated proposals to a terminal, and means for supporting a user in using furniture and interior products in a physical store to confirm and purchase the proposals. This allows the user to see, touch, and confirm the actual products in the physical store, enabling them to concretely experience the proposed interior coordination.

[0137] A "photography device" is a device used by a user to capture an image of a space, and typically refers to a smartphone or digital camera.

[0138] "Means for transmitting images and related information to the server" refers to a method for transmitting image data acquired by the photographing device and information such as the theme, budget, and color entered by the user to the server, and is exemplified by a dedicated application.

[0139] An "image analysis device" is a device or software that analyzes image data sent to a server and identifies spatial characteristics, and can use image recognition technology, for example.

[0140] A "generator" is a device or software that uses analysis results and related information to generate optimal furniture and accessory suggestions, and can incorporate the latest interior design trends in conjunction with a database.

[0141] "Means for transmitting the generated proposal to the terminal" refers to a method for transmitting the interior coordination proposal generated by the generating device to the user's terminal, and corresponds to data transmission via the Internet.

[0142] "Means to assist users in checking and purchasing suggestions using furniture and interior products in a physical store" refers to methods for supporting users in checking and purchasing provided interior suggestions using actual furniture and interior products in a physical store, and includes in-store displays and assistance from sales staff.

[0143] System Overview

[0144] The interior coordination suggestion system of this invention is based on the user capturing an image of a space using a camera and sending it along with related information to a server. The server then analyzes the received image using an image analysis device to identify the characteristics of the space. Next, a generation device generates optimal furniture and accessory suggestions based on the user's theme, budget, and color information. The generated suggestions are sent to the user's device, where the user can view and purchase the suggested items in a physical store.

[0145] Hardware and software used

[0146] 1. Imaging equipment

[0147] Images of the space are captured using a smartphone or digital camera.

[0148] 2. Server

[0149] The server acts as an API server using Flask and receives images and related information sent by users.

[0150] 3. Image analysis equipment

[0151] Image analysis software such as OpenCV and TensorFlow is used to detect spatial features (dimensions, furniture arrangement, etc.).

[0152] 4. Generator

[0153] The generator works with an internal database or an external interior design database and uses an AI model (e.g., a Transformer-based generative model) to generate optimal coordination suggestions.

[0154] 5. Terminal

[0155] A smartphone or tablet for users to view the suggestions.

[0156] Processing flow

[0157] After receiving the image and related information from the user, the server first analyzes the spatial features using an image analyzer, primarily using OpenCV to detect feature points and edges in the image, and TensorFlow to recognize objects such as furniture.

[0158] Next, the generator generates an interior coordination proposal based on the space's characteristics and the theme, budget, and color information provided by the user. The generator works in conjunction with an interior design database to make proposals that incorporate the latest designs.

[0159] The generated coordination proposals are sent to the user's device via the server, where the user can check and purchase the proposed furniture and accessories in the physical store.

[0160] Specific examples

[0161] User A takes a picture of the living room using his smartphone, and then enters information such as the theme "Scandinavian style," budget "100,000 yen," and favorite colors "blue and white" through the application and sends it.

[0162] The server uses an image analyzer to analyze the dimensions of the living room and the layout of the existing furniture, and a generator generates optimal furniture and accessory suggestions based on a Nordic theme. For example, the suggestions might look like this:

[0163] Prompt Sentence Examples

[0164] User theme: Nordic style

[0165] Budget: 100,000 yen

[0166] Favorite colors: Blue and white

[0167] The generated suggestions are sent to User A's smartphone, and User A can actually check and touch the suggested items in a physical store. User A can then purchase the products they like on the spot.

[0168] Through this process, users can create their ideal interior coordination both online and in-store.

[0169] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0170] Step 1:

[0171] The user takes a picture of their room using their smartphone. They open a dedicated application on the device, enter relevant information such as the theme (e.g., Scandinavian style), budget (e.g., 100,000 yen), and preferred colors (e.g., blue and white), and send the image and text information to the server. In this step, the input data is the captured image and text information, and the output data is generated to be sent to the server.

[0172] Step 2:

[0173] The server receives the images and related information sent by the user. On the server, an image analysis device receives the images and uses image recognition technology (e.g., OpenCV, TensorFlow) to identify spatial features (dimensions, furniture placement, etc.). In this process, the image stored on the server is used as input data, and analyzed spatial feature information is generated as output.

[0174] Step 3:

[0175] The server sends the analyzed space characteristics information and the theme, budget, and preferred color information received from the user to the generator. The generator receives this information as input data and works with an interior design database to generate optimal interior coordination proposals. Here, coordination proposals are generated using an AI model (e.g., a Transformer-based generative model) based on existing design data in the database. The generated proposal data is generated as output.

[0176] Step 4:

[0177] The server organizes the proposals returned from the generation device and sends them to the user's terminal. In this step, the generated proposal data is used as input data, and data to be sent to the user's terminal is prepared as output.

[0178] Step 5:

[0179] The user checks the proposed interior coordination on the device. The proposals are displayed through a dedicated application, and the user can actually check and touch the proposed items in a physical store. At this stage, the input is the proposal data from the server, and the output is the user's visual and tactile confirmation and purchase of the products.

[0180] Step 6:

[0181] If the user likes an item, they can proceed with the purchase process at a physical store. Support is provided for purchasing the suggested interior items on the spot, and the purchase information is processed through a dedicated application. This process takes the user's purchase intention and payment information as input data, and generates purchase completion data as output.

[0182] The above steps implement a series of operations, allowing users to experience the interior coordination in detail and then check and purchase in a physical store to realize their ideal space.

[0183] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0184] The present invention is a system in which a user acquires an image of a space, sends the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. Furthermore, by combining the present invention with an emotion engine that recognizes the user's emotions, the system can make proposals that take into account the user's emotional state. Specific embodiments are described below.

[0185] User operation

[0186] First, users take pictures of their rooms using a camera such as a smartphone. The images are then uploaded via a dedicated application. This application provides users with an interface for entering relevant information such as theme, color, and budget. The application also incorporates an emotion engine that recognizes the user's emotions, and emotional information is acquired simultaneously when the user uploads the image.

[0187] Server-side processing

[0188] The server receives the image, input information, and emotion information sent by the user. The server then sends the image data to an image analysis device and requests it to analyze the spatial features (e.g., dimensions, existing furniture layout, interior style). The image analysis device extracts the features and returns the analysis results to the server.

[0189] The analysis results and related information are sent to the generator along with the emotion information. The generator then uses this information to generate optimal interior design proposals. The generator then connects with interior design databases around the world to generate item and layout proposals based on the latest trends. Furthermore, the generator takes the user's emotional state into account and makes proposals that correspond to their emotions.

[0190] Sending generated suggestions to users

[0191] The server then organizes the suggestions returned by the generators and sends them to the user's device, where they can be viewed within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[0192] Specific examples

[0193] For example, if User B has a budget of 150,000 yen, a vintage theme, and preferred colors of brown and gold, the user can take a picture of their living room with their smartphone, enter the relevant information into a dedicated app, and send it. At this time, the emotion engine also recognizes User B's real-time emotional state and sends it to the server. The server then uses an image analysis device to analyze the dimensions of User B's living room and the layout of existing furniture, and the generator then suggests optimal furniture and accessories based on the vintage theme.

[0194] The suggestions are sent to User B's smartphone, and User B checks the suggestions within the app. If the suggested items match their emotions and they like them, User B can purchase them within the app. Through this process, users can easily create their ideal interior coordination that is in line with their emotions.

[0195] The above is a specific description of the embodiment of the present invention. This system allows users to receive optimal interior design suggestions that meet their individual needs and emotions without any effort on their part.

[0196] The processing flow will be explained below.

[0197] Step 1:

[0198] The user takes an image of the room using a camera (smartphone camera).

[0199] Step 2:

[0200] The user launches a dedicated application and uploads the captured images to the app.

[0201] Step 3:

[0202] Users input relevant information such as budget, theme, color preferences, etc. through the application interface, and the emotion engine automatically captures emotional information from the user's facial expressions and voice.

[0203] Step 4:

[0204] The user presses the "send" button to send the image, input information, and emotion information to the server.

[0205] Step 5:

[0206] The server receives the image, input information, and emotion information sent by the user.

[0207] Step 6:

[0208] The server sends the received image data to an image analysis device, which analyzes the characteristics of the space (dimensions, arrangement of existing furniture, interior style, etc.).

[0209] Step 7:

[0210] The image analysis device performs the analysis and returns the results to the server.

[0211] Step 8:

[0212] The server transmits the analysis results, information about the theme, budget, and color preferences entered by the user, and emotional information to the generation device.

[0213] Step 9:

[0214] The generator generates optimal furniture and accessory recommendations based on the analysis results and user input. The generator also takes into account the user's emotional information to generate recommendations that are appropriate to their emotions. The generator also connects to a global interior design database, enabling it to provide recommendations based on the latest trends.

[0215] Step 10:

[0216] The generator sends the generated proposal back to the server.

[0217] Step 11:

[0218] The server organizes the proposals and sends them to the user's device.

[0219] Step 12:

[0220] The user checks the proposal on their device (smartphone), which includes a new furniture layout, a list of recommended items, and a link to purchase.

[0221] Step 13:

[0222] The user reviews the suggestions and selects the items they like. The user completes the purchase within the app.

[0223] This allows users to receive basic interior coordination suggestions as well as emotional suggestions. For example, if a user wants to relax, the app will suggest furniture and accessories in calming colors.

[0224] Example 2

[0225] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0226] Conventional interior design suggestion systems have difficulty making suggestions that fully consider the user's individual needs and emotional state. Another problem is the lack of technology that can accurately grasp the characteristics of a space based on image analysis and generate optimal interior design suggestions based on that information. This means that users have to spend a lot of time selecting the interior design themselves, and are unable to obtain satisfactory suggestions.

[0227] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0228] In this invention, the server includes means for capturing images of the space using a photographing device, means for transmitting the images and related information to the server, means for identifying characteristics of the space using an image analyzing device, means for providing a generating device for generating optimal furniture and accessory proposals, means for transmitting the generated proposals to the terminal, and means for acquiring the user's emotional information using an emotion engine and reflecting the information in the generated proposals. This enables the user to easily receive optimal interior proposals that match their individual needs and emotional state based on the captured images, related information, and emotional information.

[0229] A "photography device" is a device that a user uses to capture an image of a space, and typically refers to a smartphone or digital camera equipped with a camera function.

[0230] "Means for transmitting images and related information to a server" refers to a function for transmitting images acquired by a user and information such as theme, budget, and preferred colors to a server via the Internet.

[0231] An "image analysis device" refers to software or hardware that analyzes received image data and extracts spatial characteristics (dimensions, furniture arrangement, interior style, etc.).

[0232] A "generator" is a system that generates optimal furniture and accessory suggestions based on image analysis results and related information, and is typically implemented using a generative AI model.

[0233] The "means for transmitting the generated proposal to the terminal" refers to a communication function for transmitting the interior proposal generated by the generation device to the user's terminal.

[0234] An "emotion engine" is a system for analyzing and acquiring the user's emotional state, and for example, recognizes the user's emotions using facial expression recognition technology.

[0235] The "Interior Design Database" is a database that accumulates data on interior design from around the world and provides information on the latest trends and items.

[0236] MODE FOR CARRYING OUT THE INVENTION

[0237] The present invention is a system in which a user acquires an image of a space, sends the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. Furthermore, by combining the present invention with an emotion engine that recognizes the user's emotions, it is possible to make proposals that take into account the user's emotional state. Specific embodiments are described below.

[0238] User operation

[0239] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded to a server via a dedicated application. This application provides the user with an interface for entering relevant information such as theme, color, and budget. The application also incorporates an emotion engine, which simultaneously acquires emotional information when the user uploads the image. The emotion engine uses facial expression recognition technology to recognize the user's emotions in real time.

[0240] Server-side processing

[0241] The server receives the image, related information, and emotional information sent by the user. The server then sends the image data to an image analyzer (e.g., an image analysis library such as OpenCV or TensorFlow) to analyze the spatial features (e.g., dimensions, existing furniture arrangement, interior style, etc.). The image analyzer then uses a deep learning model to extract these features from the image.

[0242] The server then sends the analysis results, related information, and emotional information to a generator (such as OpenAI's (registered trademark) GPT-4, for example) to generate optimal interior design suggestions. The generator connects with a global interior design database to generate item and layout suggestions based on the latest trends. Furthermore, it makes suggestions that take the user's emotional state into account.

[0243] Sending generated suggestions to users

[0244] The server then compiles the suggestions returned by the generators and sends them to the user's device, where they can view them within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[0245] Specific examples

[0246] For example, if User B sets a budget of 150,000 yen, a vintage theme, and favorite colors of brown and gold, User B takes a picture of their living room with their smartphone, enters the image and related information into a dedicated app, and sends it. At this time, the emotion engine built into the app recognizes User B's emotional state in real time and sends it to the server.

[0247] The server uses OpenCV to analyze the dimensions and existing furniture layout of User B's living room. The analysis results are sent back to the server, which then uses this information to generate optimal furniture and accessory recommendations based on a vintage theme using tools such as OpenAI's GPT-4. Once the recommendations are sent to User B's smartphone, User B can review the recommendations—for example, a vintage-style sofa, lamp, or carpet—in the app, and if they like them, they can simply tap the purchase link to purchase them.

[0248] Prompt Sentence Examples

[0249] "I'd like some suggestions for a vintage-inspired interior. My budget is ¥150,000, my favorite colors are brown and gold, and my current emotional state is relaxed. The image of my living room includes my current furniture arrangement. Please suggest the best arrangement and items."

[0250] The above is a specific embodiment of the present invention. This system allows users to easily create an ideal interior coordination that is in line with their emotions.

[0251] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0252] Step 1:

[0253] Taking photos and uploading images

[0254] input:

[0255] The user takes a picture of their room using the smartphone camera.

[0256] Specific behavior:

[0257] The user opens the camera app on their smartphone and takes a picture that shows the entire room. Then, they open the dedicated application and tap the "Upload" button to import the image into the application.

[0258] output:

[0259] Image data stored on the smartphone is sent to the server via a dedicated application.

[0260] Step 2:

[0261] Enter and submit relevant information

[0262] input:

[0263] Users fill out forms within the application to enter relevant information such as theme, color, budget, etc.

[0264] Specific behavior:

[0265] Users enter information such as the theme (e.g., vintage style), preferred colors (e.g., brown and gold), and budget (e.g., 150,000 yen) into the application's input form and tap the "Submit" button.

[0266] output:

[0267] Relevant information such as theme, color, budget, etc. is sent to the server and associated with the image data.

[0268] Step 3:

[0269] Acquiring emotional information

[0270] input:

[0271] When users upload images, emotional information is collected using facial recognition technology.

[0272] Specific behavior:

[0273] An emotion engine built into the application analyzes the user's face in real time to detect their emotional state (e.g., relaxed).

[0274] output:

[0275] Emotional state data is generated and transmitted to a server.

[0276] Step 4:

[0277] Data reception and analysis

[0278] input:

[0279] The server receives the image data, related information (theme, color, budget), and emotional information sent by the user.

[0280] Specific behavior:

[0281] The server receives the captured image data, related information, and emotion information and temporarily stores them in a database.

[0282] output:

[0283] Stored image data and related information, as well as emotional information.

[0284] Step 5:

[0285] Image analysis

[0286] input:

[0287] Image data stored on the server.

[0288] Specific behavior:

[0289] The server uses image analysis equipment (e.g., OpenCV or TensorFlow) to analyze the image data and extract spatial features (e.g., dimensions, existing furniture arrangement, interior style).

[0290] output:

[0291] Extracted spatial feature data.

[0292] Step 6:

[0293] Proposal generation

[0294] input:

[0295] Analysis results, related information (theme, color, budget) and sentiment information stored on the server.

[0296] Specific behavior:

[0297] The server sends the analysis results, related information, and emotional information to a generator (e.g., OpenAI's GPT-4) to generate optimal interior design proposals. The generator connects to an interior design database to generate item and layout proposals based on the latest interior trends.

[0298] output:

[0299] Generated interior design proposal data (furniture layout, list of recommended items, and purchase links).

[0300] Step 7:

[0301] Submitting generated proposals

[0302] input:

[0303] Generated interior proposal data.

[0304] Specific behavior:

[0305] The server organizes the generated proposals and sends them to the user's terminal.

[0306] output:

[0307] Interior design suggestions displayed on the user's device.

[0308] Step 8:

[0309] Confirmation and decision on proposal contents

[0310] input:

[0311] The offer received by the user's device.

[0312] Specific behavior:

[0313] Users can review the suggestions within the app, select the one they like, and tap the purchase link.

[0314] output:

[0315] Based on the user's purchasing intent, the user is redirected to a page where they can purchase the suggested furniture and accessories.

[0316] In this way, users can easily and quickly receive interior design suggestions that suit their needs and emotions.

[0317] (Application example 2)

[0318] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0319] In modern interior design, users face the challenge of finding suggestions that reflect their emotions and immediate mood. It is also difficult for users to directly and intuitively understand where suggested items will be placed when selecting products in a physical store. Therefore, it is necessary to provide more personalized interior design suggestions by analyzing users' real-time emotions and states and guiding the location of offered items in a physical store.

[0320] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring an image of the space using an imaging device, means for transmitting the image and related information to the server, means for identifying characteristics of the space using an image analysis device in the server, means for including a generation device that generates optimal furniture and accessory suggestions using the analysis results and related information, means for including an emotion engine that recognizes the user's emotional state and adjusts the suggestions based on the emotion information, and means for transmitting the generated suggestions to a terminal and guiding the location of recommended items. This allows the user to receive interior suggestions tailored to their emotional state in real time and be guided in selecting the locations of specific items in a physical store.

[0321] An "imaging device" is a device that allows a user to capture an image of a space, and includes a smartphone, smart glasses, etc.

[0322] "Related information" refers to information related to interior coordination such as budget, theme, and color provided by the user.

[0323] A "server" is a computer system that receives images and information sent by users and performs analysis and generation.

[0324] An "image analysis device" is a device that analyzes transmitted images and identifies the characteristics, dimensions, existing furniture layout, etc. of a space.

[0325] A "generator" is a device that creates optimal furniture and accessory proposals based on analysis results and related information.

[0326] An "emotion engine" is a system that recognizes emotions from a user's facial expressions, voice, etc. and analyzes that information.

[0327] "Terminal" refers to a device that receives and displays interior design proposals generated by a user, and includes smartphones, smart glasses, etc.

[0328] "Means for guiding the location of recommended items" refers to a function that guides the user to the specific location of recommended products or items in the generated suggestions.

[0329] The "interior design database" is a database that stores information on various interior styles and trends, and is a data source that the generation device references when generating proposals.

[0330] In this invention, a user wears smart glasses and walks around a physical store, photographing the products and space inside the store. The smart glasses capture the scenery and products the user sees in real time and save them as images. The user can also input relevant information such as budget, theme, and color preferences into the smart glasses using voice input. At this time, the smart glasses are equipped with an emotion engine that analyzes the user's emotional state from facial expressions and tone of voice to acquire emotional information.

[0331] The server receives the images, related information, and emotion information sent by the user. The server then uses an image analyzer to identify spatial features (dimensions, existing furniture layout, etc.) from the received images. Software libraries such as OpenCV are used for image analysis, and machine learning models such as TensorFlow are used for emotion analysis.

[0332] The server generates optimal interior design suggestions based on the analysis results, related information, and emotional information. The generator is connected to a global interior design database and can suggest furniture and accessories based on the latest trends. The suggestions are adjusted to suit the user's emotional state.

[0333] The generated suggestions are sent to the user's smart glasses in real time. The smart glasses display guides the user to the location of the recommended items, making it easier for the user to identify them in the store. This allows users to easily coordinate interiors that match their emotions and preferences in a physical store.

[0334] Specific examples

[0335] As the user wears the smart glasses and walks around an interior design shop, the glasses take photos of the furniture and decorations in the store. The user inputs "budget 200,000 yen, theme modern, color preference black and white" by voice, and the emotion engine analyzes the user's real-time emotional state and sends it to the server. The server uses an image analysis device to analyze the dimensions and placement of products in the store, and a generator recommends modern-themed furniture and accessories that match the user's emotional information. The glasses' display shows the location of the recommended item along with the message "This item is perfect for your mood!"

[0336] Prompt statement

[0337] "A user visits an interior design shop and uses the smart glasses to input information such as a budget of 200,000 yen, a modern theme, and a color preference of black and white, all while looking around the shop. The smart glasses also analyze the user's emotions and suggest the optimal interior coordination. The suggestions are displayed on the glasses' display in real time, and the user is guided to the location of items based on the suggestions."

[0338] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0339] Step 1:

[0340] A user wears smart glasses and walks around a physical store. The smart glasses' camera captures images of the store in real time. The user inputs relevant information, such as budget, theme, and color preferences, into the smart glasses using voice input. The smart glasses' camera captures image data, and the voice input device captures the relevant information as text. These data are temporarily stored in the smart glasses' memory.

[0341] Input: In-store images, relevant information such as the user's budget, theme, colors, etc.

[0342] Output: Image data, related text data

[0343] Step 2:

[0344] The emotion engine built into the smart glasses analyzes the user's emotional state in real time from their facial expressions and tone of voice. The emotion engine generates emotional information using an AI model that combines facial expression recognition and voice analysis, and stores this information in the smart glasses. The user's emotional information is also generated as text data.

[0345] Input: User's facial expression, tone of voice

[0346] Output: Emotional information text data

[0347] Step 3:

[0348] The smart glasses transmit the acquired image data, related information, and emotional information to a server via wireless communication (Wi-Fi or Bluetooth). The server receives and stores this data.

[0349] Input: Image data, text data of related information, text data of emotional information

[0350] Output: Data stored on the server

[0351] Step 4:

[0352] The server uses an image analysis device to analyze the received image data and identify the characteristics of the space. Software libraries such as OpenCV are used for image analysis. The analysis results, including the dimensions of the space and the layout of existing furniture, are generated as text data and saved on the server.

[0353] Input: Image data

[0354] Output: Text data of spatial features (dimensions, furniture layout, etc.)

[0355] Step 5:

[0356] The server generates optimal interior design proposals based on the analysis results, related information, and emotional information. The generator connects to a global interior design database, which generates furniture and accessory proposals based on the latest trends. The proposals are adjusted to suit the user's emotional state. The final proposals are generated as text and image data.

[0357] Input: Text data of spatial features, text data of related information, text data of emotional information

[0358] Output: Text data and image data of optimal interior design proposals

[0359] Step 6:

[0360] The server sends the generated interior design proposals to the smart glasses via wireless communication (Wi-Fi or Bluetooth). The smart glasses receive the proposals and store them in their internal memory.

[0361] Input: Text data and image data of optimal interior proposals

[0362] Output: Proposal data stored in smart glasses

[0363] Step 7:

[0364] The smart glasses display the suggested items on a screen. The user is guided to locate the suggested items in a physical store. Augmented reality (AR) technology is used to display the guidance. The location of the suggested item is overlaid on the display, and the user can navigate to that location.

[0365] Input: Proposal data stored in smart glasses

[0366] Output: Suggestions and guidance displayed on the smart glasses display

[0367] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0368] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0369] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0370] [Second embodiment]

[0371] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0372] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0373] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0374] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0375] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0376] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0377] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0378] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0379] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0380] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0381] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0382] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0383] The present invention is a system in which a user acquires an image of a space, transmits the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. A specific embodiment of this system is described below.

[0384] User operation

[0385] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded via a dedicated application. This application provides the user with an interface for entering relevant information such as theme, color, and budget. The user enters this information and presses the "send" button to send the data to the server.

[0386] Server-side processing

[0387] The server receives the image and input information sent by the user. The server then sends the image data to an image analysis device, which analyzes the space's characteristics (e.g., dimensions, existing furniture layout, interior style). The analysis device extracts these characteristics and returns the analysis results to the server.

[0388] The server then sends these analysis results, along with information entered by the user such as theme, color, and budget, to the generator, which then generates optimal interior coordination proposals. The generator connects to interior design databases from around the world and can generate optimal item and layout proposals based on the latest trends.

[0389] Sending generated suggestions to users

[0390] The server then organizes the suggestions returned by the generators and sends them to the user's device, where they can be viewed within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[0391] Specific examples

[0392] For example, if User A has a budget of 100,000 yen, a Nordic theme, and favorite colors of blue and white, the user takes a picture of their living room with their smartphone, enters the relevant information into a dedicated app, and submits it. The server uses an image analysis device to analyze the dimensions of User A's living room and the layout of existing furniture, and the generator then proposes optimal furniture and accessories based on the Nordic theme.

[0393] The proposals are sent to User A's smartphone, and User A can check the proposals within the app. If User A likes the proposed items, he or she can purchase them within the app. Through this series of processes, users can easily realize their ideal interior coordination.

[0394] The above is a specific description of the embodiment of the present invention. This system allows users to receive optimal interior design proposals that meet their individual needs without any effort on their part.

[0395] The processing flow will be explained below.

[0396] Step 1:

[0397] The user takes an image of the room using a camera (smartphone camera).

[0398] Step 2:

[0399] The user launches a dedicated application and uploads the captured images to the app.

[0400] Step 3:

[0401] The user inputs relevant information such as budget, theme, color preferences, etc. through the application interface.

[0402] Step 4:

[0403] The user presses the "send" button to send the image and input information to the server.

[0404] Step 5:

[0405] The server receives the image and input information sent by the user.

[0406] Step 6:

[0407] The server sends the image data to an image analysis device, which analyzes the characteristics of the space (dimensions, existing furniture layout, interior style).

[0408] Step 7:

[0409] The image analyzer analyzes the image and extracts spatial features, and the results are sent back to the server.

[0410] Step 8:

[0411] The server transmits the analysis results and user input information to the generation device, which then generates an optimal interior proposal.

[0412] Step 9:

[0413] The generator generates optimal furniture and accessory proposals based on the analysis results and user input, linking with interior design databases around the world.

[0414] Step 10:

[0415] The server transmits the proposal content returned from the generating device to the user's terminal.

[0416] Step 11:

[0417] The user checks the proposal on their device (smartphone), which includes a furniture layout diagram, a list of recommended items, and a link to purchase.

[0418] Step 12:

[0419] The user selects the suggested item and checks out within the app.

[0420] This allows users to easily create their ideal interior coordination.

[0421] Example 1

[0422] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0423] Conventional interior coordination suggestion systems have the drawback of making it difficult for users to obtain proposals that match their preferences and budget, resulting in a time-consuming and labor-intensive process. Furthermore, obtaining optimal proposals that meet the user's needs requires advanced analysis and generation technology, but such technology is often not fully utilized.

[0424] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0425] In this invention, the server includes means for acquiring images of the space using a photographing device, means for transmitting the images and related information to the server, means for identifying the characteristics of the space using an image analyzing device in the server, means for providing a generating device for generating optimal furniture and accessory proposals using a generative AI model based on the analysis results and related information, and means for transmitting the generated proposals to a terminal. This enables a user to quickly and accurately receive interior coordination proposals based on the user's photographed images and desired conditions.

[0426] "Capture device" refers to a device for capturing images of a space. Examples include smartphones and digital cameras.

[0427] "Server" refers to a computer system that receives, stores, and processes data over a network.

[0428] "Image analysis device" refers to software or hardware for extracting and analyzing features from images. Specific examples include TensorFlow and OpenCV.

[0429] "Generative AI models" refer to artificial intelligence models that generate optimal suggestions based on user criteria. Specific examples include GPT-4.

[0430] "Terminal" refers to a device that receives data from a server and is used as an interface by a user. Examples include smartphones and tablets.

[0431] "Spatial characteristics" refers to information such as the dimensions of the space, existing furniture layout, and interior style obtained through image analysis.

[0432] "Related information" refers to information entered by the user, such as theme, favorite color, budget, etc.

[0433] "Furniture and accessory suggestions" refers to specific interior layout ideas and lists of recommended items generated based on the user's criteria and analysis results.

[0434] "Database" refers to a system for storing and managing interior design data and other related information.

[0435] The present invention is a system in which a user acquires an image of a space, transmits the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. A specific embodiment of this system is described below.

[0436] User operation

[0437] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded via a dedicated application. This application provides the user with an interface for entering relevant information such as the theme, favorite colors, and budget. The user enters this information and presses the "send" button to send the data to the server.

[0438] Server-side processing

[0439] The server receives the image and related information sent by the user. The server then sends the image data to an image analyzer, requesting it to analyze the spatial features (e.g., dimensions, existing furniture layout, interior style). The image analyzer can use software such as TensorFlow or OpenCV. The analyzer extracts these features and returns the analysis results to the server.

[0440] The server then uses a generative AI model (such as GPT-4) to generate optimal interior coordination suggestions based on these analysis results and user information. The generator connects to interior design databases around the world to make suggestions based on the latest trends.

[0441] Sending generated suggestions to users

[0442] The server then organizes the suggestions returned by the generator and sends them to the user's device. The suggestions include a detailed furniture layout plan, a list of recommended items, and links to purchase them. The user can view these suggestions in a dedicated application.

[0443] Specific examples

[0444] For example, the specific operating procedure for User A, who has a budget of 100,000 yen, a Nordic theme, and preferred colors of blue and white, is shown below. User A first takes a picture of their living room with their smartphone and uploads it to a dedicated app. Next, they enter the relevant information into the app's interface and send it to the server. The server uses TensorFlow to analyze the dimensions and existing furniture layout of User A's living room, and uses GPT-4 to generate optimal interior design suggestions. The generated suggestions are sent to User A's smartphone, where User A can view the suggestions within the app.

[0445] Prompt Sentence Examples

[0446] In the case of User A, the generative AI model receives the following prompt: "Please suggest an interior design with a Nordic theme and blue and white colors for a budget of 100,000 yen."

[0447] In this way, by using the system of the present invention, the user can easily achieve the ideal interior coordination.

[0448] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0449] Processing Steps

[0450] Step 1:

[0451] The user takes a picture and sends it to the server through the application.

[0452] Input: User-taken images of your space, along with related information like theme, favorite colors, budget, etc.

[0453] How it works: The user takes a picture of their room using the smartphone camera, uploads it to a dedicated application, and then enters the theme, favorite colors, and budget information through the application interface and presses the send button to send it to the server.

[0454] Step 2:

[0455] A server receives the image and related information.

[0456] Output: Storage of image data and related information sent to the server

[0457] Specific operation: The server receives the image file, theme, favorite color, and budget information sent by the user, and stores them in a database in JSON format. The received data is linked to the user ID.

[0458] Step 3:

[0459] The server sends the image data to an image analysis device, which analyzes the characteristics of the space.

[0460] Input: Image data stored on the server

[0461] Output: Analysis results such as space dimensions, existing furniture arrangement, and interior style

[0462] How it works: The server sends the stored images to an image analyzer (e.g., TensorFlow or OpenCV), which extracts feature information from the images, such as the dimensions of the space, the existing furniture layout, and the interior style, and returns the analysis results to the server.

[0463] Step 4:

[0464] The server transmits the analysis results and user information to the generation device.

[0465] Input: Analysis results and related information entered by the user (theme, color, budget)

[0466] Output: Interior coordination suggestions generated by the generative AI model

[0467] Specific operation: The server combines the analysis results with the user's relevant information to create and send a prompt to the generative AI model (e.g., GPT-4). An example prompt is "Please suggest an interior coordination with a Nordic theme and blue and white colors for a budget of 100,000 yen." The generative AI model then generates the optimal interior coordination proposal and sends it back to the server.

[0468] Step 5:

[0469] The server organizes the generated results and sends them to the user terminal.

[0470] Input: Coordinate proposals from the generator

[0471] Output: Sending the organized proposals to the user's device

[0472] Specific operation: The server organizes the proposals received from the generator, formats them appropriately, and sends them to the user's device. The proposals include a detailed furniture layout plan, a list of recommended items, and links to purchase them.

[0473] Step 6:

[0474] The user reviews the suggestions within the application and purchases the items if necessary.

[0475] Input: Interior coordination proposal sent from the server

[0476] What it does: The user opens the app and sees the suggestions sent from the server, including a detailed furniture layout and a list of recommended items, and can purchase the items they like right within the app.

[0477] The above is the specific processing flow of the program for this system, as well as the input, output, and specific operation at each step.

[0478] (Application example 1)

[0479] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0480] Conventional interior coordination systems only allow users to check the proposed interior designs online, making it difficult for them to actually use and check the products. As a result, users are unable to check the actual appearance and texture of the products before purchasing them, which has the problem of not being able to fully stimulate users' purchasing motivation.

[0481] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0482] In this invention, the server includes means for capturing an image of the space using a photographing device, means for transmitting the image and related information to the server, means for identifying characteristics of the space using an image analyzing device in the server, means for providing a generating device for generating optimal furniture and accessory proposals using the analysis results and related information, means for transmitting the generated proposals to a terminal, and means for supporting a user in using furniture and interior products in a physical store to confirm and purchase the proposals. This allows the user to see, touch, and confirm the actual products in the physical store, enabling them to concretely experience the proposed interior coordination.

[0483] A "photography device" is a device used by a user to capture an image of a space, and typically refers to a smartphone or digital camera.

[0484] "Means for transmitting images and related information to the server" refers to a method for transmitting image data acquired by the photographing device and information such as the theme, budget, and color entered by the user to the server, and is exemplified by a dedicated application.

[0485] An "image analysis device" is a device or software that analyzes image data sent to a server and identifies spatial characteristics, and can use image recognition technology, for example.

[0486] A "generator" is a device or software that uses analysis results and related information to generate optimal furniture and accessory suggestions, and can incorporate the latest interior design trends in conjunction with a database.

[0487] "Means for transmitting the generated proposal to the terminal" refers to a method for transmitting the interior coordination proposal generated by the generating device to the user's terminal, and corresponds to data transmission via the Internet.

[0488] "Means to assist users in checking and purchasing suggestions using furniture and interior products in a physical store" refers to methods for supporting users in checking and purchasing provided interior suggestions using actual furniture and interior products in a physical store, and includes in-store displays and assistance from sales staff.

[0489] System Overview

[0490] The interior coordination suggestion system of this invention is based on the user capturing an image of a space using a camera and sending it along with related information to a server. The server then analyzes the received image using an image analysis device to identify the characteristics of the space. Next, a generation device generates optimal furniture and accessory suggestions based on the user's theme, budget, and color information. The generated suggestions are sent to the user's device, where the user can view and purchase the suggested items in a physical store.

[0491] Hardware and software used

[0492] 1. Imaging equipment

[0493] Images of the space are captured using a smartphone or digital camera.

[0494] 2. Server

[0495] The server acts as an API server using Flask and receives images and related information sent by users.

[0496] 3. Image analysis equipment

[0497] Image analysis software such as OpenCV and TensorFlow is used to detect spatial features (dimensions, furniture arrangement, etc.).

[0498] 4. Generator

[0499] The generator works with an internal database or an external interior design database and uses an AI model (e.g., a Transformer-based generative model) to generate optimal coordination suggestions.

[0500] 5. Terminal

[0501] A smartphone or tablet for users to view the suggestions.

[0502] Processing flow

[0503] After receiving the image and related information from the user, the server first analyzes the spatial features using an image analyzer, primarily using OpenCV to detect feature points and edges in the image, and TensorFlow to recognize objects such as furniture.

[0504] Next, the generator generates an interior coordination proposal based on the space's characteristics and the theme, budget, and color information provided by the user. The generator works in conjunction with an interior design database to make proposals that incorporate the latest designs.

[0505] The generated coordination proposals are sent to the user's device via the server, where the user can check and purchase the proposed furniture and accessories in the physical store.

[0506] Specific examples

[0507] User A takes a picture of the living room using his smartphone, and then enters information such as the theme "Scandinavian style," budget "100,000 yen," and favorite colors "blue and white" through the application and sends it.

[0508] The server uses an image analyzer to analyze the dimensions of the living room and the layout of the existing furniture, and a generator generates optimal furniture and accessory suggestions based on a Nordic theme. For example, the suggestions might look like this:

[0509] Prompt Sentence Examples

[0510] User theme: Nordic style

[0511] Budget: 100,000 yen

[0512] Favorite colors: Blue and white

[0513] The generated suggestions are sent to User A's smartphone, and User A can actually check and touch the suggested items in a physical store. User A can then purchase the products they like on the spot.

[0514] Through this process, users can create their ideal interior coordination both online and in-store.

[0515] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0516] Step 1:

[0517] The user takes a picture of their room using their smartphone. They open a dedicated application on the device, enter relevant information such as the theme (e.g., Scandinavian style), budget (e.g., 100,000 yen), and preferred colors (e.g., blue and white), and send the image and text information to the server. In this step, the input data is the captured image and text information, and the output data is generated to be sent to the server.

[0518] Step 2:

[0519] The server receives the images and related information sent by the user. On the server, an image analysis device receives the images and uses image recognition technology (e.g., OpenCV, TensorFlow) to identify spatial features (dimensions, furniture placement, etc.). In this process, the image stored on the server is used as input data, and analyzed spatial feature information is generated as output.

[0520] Step 3:

[0521] The server sends the analyzed space characteristics information and the theme, budget, and preferred color information received from the user to the generator. The generator receives this information as input data and works with an interior design database to generate optimal interior coordination proposals. Here, coordination proposals are generated using an AI model (e.g., a Transformer-based generative model) based on existing design data in the database. The generated proposal data is generated as output.

[0522] Step 4:

[0523] The server organizes the proposals returned from the generation device and sends them to the user's terminal. In this step, the generated proposal data is used as input data, and data to be sent to the user's terminal is prepared as output.

[0524] Step 5:

[0525] The user checks the proposed interior coordination on the device. The proposals are displayed through a dedicated application, and the user can actually check and touch the proposed items in a physical store. At this stage, the input is the proposal data from the server, and the output is the user's visual and tactile confirmation and purchase of the products.

[0526] Step 6:

[0527] If the user likes an item, they can proceed with the purchase process at a physical store. Support is provided for purchasing the suggested interior items on the spot, and the purchase information is processed through a dedicated application. This process takes the user's purchase intention and payment information as input data, and generates purchase completion data as output.

[0528] The above steps implement a series of operations, allowing users to experience the interior coordination in detail and then check and purchase in a physical store to realize their ideal space.

[0529] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0530] The present invention is a system in which a user acquires an image of a space, sends the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. Furthermore, by combining the present invention with an emotion engine that recognizes the user's emotions, the system can make proposals that take into account the user's emotional state. Specific embodiments are described below.

[0531] User operation

[0532] First, users take pictures of their rooms using a camera such as a smartphone. The images are then uploaded via a dedicated application. This application provides users with an interface for entering relevant information such as theme, color, and budget. The application also incorporates an emotion engine that recognizes the user's emotions, and emotional information is acquired simultaneously when the user uploads the image.

[0533] Server-side processing

[0534] The server receives the image, input information, and emotion information sent by the user. The server then sends the image data to an image analysis device and requests it to analyze the spatial features (e.g., dimensions, existing furniture layout, interior style). The image analysis device extracts the features and returns the analysis results to the server.

[0535] The analysis results and related information are sent to the generator along with the emotion information. The generator then uses this information to generate optimal interior design proposals. The generator then connects with interior design databases around the world to generate item and layout proposals based on the latest trends. Furthermore, the generator takes the user's emotional state into account and makes proposals that correspond to their emotions.

[0536] Sending generated suggestions to users

[0537] The server then organizes the suggestions returned by the generators and sends them to the user's device, where they can be viewed within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[0538] Specific examples

[0539] For example, if User B has a budget of 150,000 yen, a vintage theme, and preferred colors of brown and gold, the user can take a picture of their living room with their smartphone, enter the relevant information into a dedicated app, and send it. At this time, the emotion engine also recognizes User B's real-time emotional state and sends it to the server. The server then uses an image analysis device to analyze the dimensions of User B's living room and the layout of existing furniture, and the generator then suggests optimal furniture and accessories based on the vintage theme.

[0540] The suggestions are sent to User B's smartphone, and User B checks the suggestions within the app. If the suggested items match their emotions and they like them, User B can purchase them within the app. Through this process, users can easily create their ideal interior coordination that is in line with their emotions.

[0541] The above is a specific description of the embodiment of the present invention. This system allows users to receive optimal interior design suggestions that meet their individual needs and emotions without any effort on their part.

[0542] The processing flow will be explained below.

[0543] Step 1:

[0544] The user takes an image of the room using a camera (smartphone camera).

[0545] Step 2:

[0546] The user launches a dedicated application and uploads the captured images to the app.

[0547] Step 3:

[0548] Users input relevant information such as budget, theme, color preferences, etc. through the application interface, and the emotion engine automatically captures emotional information from the user's facial expressions and voice.

[0549] Step 4:

[0550] The user presses the "send" button to send the image, input information, and emotion information to the server.

[0551] Step 5:

[0552] The server receives the image, input information, and emotion information sent by the user.

[0553] Step 6:

[0554] The server sends the received image data to an image analysis device, which analyzes the characteristics of the space (dimensions, arrangement of existing furniture, interior style, etc.).

[0555] Step 7:

[0556] The image analysis device performs the analysis and returns the results to the server.

[0557] Step 8:

[0558] The server transmits the analysis results, information about the theme, budget, and color preferences entered by the user, and emotional information to the generation device.

[0559] Step 9:

[0560] The generator generates optimal furniture and accessory recommendations based on the analysis results and user input. The generator also takes into account the user's emotional information to generate recommendations that are appropriate to their emotions. The generator also connects to a global interior design database, enabling it to provide recommendations based on the latest trends.

[0561] Step 10:

[0562] The generator sends the generated proposal back to the server.

[0563] Step 11:

[0564] The server organizes the proposals and sends them to the user's device.

[0565] Step 12:

[0566] The user checks the proposal on their device (smartphone), which includes a new furniture layout, a list of recommended items, and a link to purchase.

[0567] Step 13:

[0568] The user reviews the suggestions and selects the items they like. The user completes the purchase within the app.

[0569] This allows users to receive basic interior coordination suggestions as well as emotional suggestions. For example, if a user wants to relax, the app will suggest furniture and accessories in calming colors.

[0570] Example 2

[0571] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0572] Conventional interior design suggestion systems have difficulty making suggestions that fully consider the user's individual needs and emotional state. Another problem is the lack of technology that can accurately grasp the characteristics of a space based on image analysis and generate optimal interior design suggestions based on that information. This means that users have to spend a lot of time selecting the interior design themselves, and are unable to obtain satisfactory suggestions.

[0573] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0574] In this invention, the server includes means for capturing images of the space using a photographing device, means for transmitting the images and related information to the server, means for identifying characteristics of the space using an image analyzing device, means for providing a generating device for generating optimal furniture and accessory proposals, means for transmitting the generated proposals to the terminal, and means for acquiring the user's emotional information using an emotion engine and reflecting the information in the generated proposals. This enables the user to easily receive optimal interior proposals that match their individual needs and emotional state based on the captured images, related information, and emotional information.

[0575] A "photography device" is a device that a user uses to capture an image of a space, and typically refers to a smartphone or digital camera equipped with a camera function.

[0576] "Means for transmitting images and related information to a server" refers to a function for transmitting images acquired by a user and information such as theme, budget, and preferred colors to a server via the Internet.

[0577] An "image analysis device" refers to software or hardware that analyzes received image data and extracts spatial characteristics (dimensions, furniture arrangement, interior style, etc.).

[0578] A "generator" is a system that generates optimal furniture and accessory suggestions based on image analysis results and related information, and is typically implemented using a generative AI model.

[0579] The "means for transmitting the generated proposal to the terminal" refers to a communication function for transmitting the interior proposal generated by the generation device to the user's terminal.

[0580] An "emotion engine" is a system for analyzing and acquiring the user's emotional state, and for example, recognizes the user's emotions using facial expression recognition technology.

[0581] The "Interior Design Database" is a database that accumulates data on interior design from around the world and provides information on the latest trends and items.

[0582] MODE FOR CARRYING OUT THE INVENTION

[0583] The present invention is a system in which a user acquires an image of a space, sends the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. Furthermore, by combining the present invention with an emotion engine that recognizes the user's emotions, it is possible to make proposals that take into account the user's emotional state. Specific embodiments are described below.

[0584] User operation

[0585] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded to a server via a dedicated application. This application provides the user with an interface for entering relevant information such as theme, color, and budget. The application also incorporates an emotion engine, which simultaneously acquires emotional information when the user uploads the image. The emotion engine uses facial expression recognition technology to recognize the user's emotions in real time.

[0586] Server-side processing

[0587] The server receives the image, related information, and emotional information sent by the user. The server then sends the image data to an image analyzer (e.g., an image analysis library such as OpenCV or TensorFlow) to analyze the spatial features (e.g., dimensions, existing furniture arrangement, interior style, etc.). The image analyzer then uses a deep learning model to extract these features from the image.

[0588] The server then sends the analysis results, related information, and emotional information to a generator (such as OpenAI's GPT-4, an example of a generative AI model) to generate optimal interior design suggestions. The generator connects with interior design databases around the world to generate item and layout suggestions based on the latest trends. It also makes suggestions that take the user's emotional state into account.

[0589] Sending generated suggestions to users

[0590] The server then compiles the suggestions returned by the generators and sends them to the user's device, where they can view them within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[0591] Specific examples

[0592] For example, if User B sets a budget of 150,000 yen, a vintage theme, and favorite colors of brown and gold, User B takes a picture of their living room with their smartphone, enters the image and related information into a dedicated app, and sends it. At this time, the emotion engine built into the app recognizes User B's emotional state in real time and sends it to the server.

[0593] The server uses OpenCV to analyze the dimensions and existing furniture layout of User B's living room. The analysis results are sent back to the server, which then uses this information to generate optimal furniture and accessory recommendations based on a vintage theme using tools such as OpenAI's GPT-4. Once the recommendations are sent to User B's smartphone, User B can review the recommendations—for example, a vintage-style sofa, lamp, or carpet—in the app, and if they like them, they can simply tap the purchase link to purchase them.

[0594] Prompt Sentence Examples

[0595] "I'd like some suggestions for a vintage-inspired interior. My budget is ¥150,000, my favorite colors are brown and gold, and my current emotional state is relaxed. The image of my living room includes my current furniture arrangement. Please suggest the best arrangement and items."

[0596] The above is a specific embodiment of the present invention. This system allows users to easily create an ideal interior coordination that is in line with their emotions.

[0597] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0598] Step 1:

[0599] Taking photos and uploading images

[0600] input:

[0601] The user takes a picture of their room using the smartphone camera.

[0602] Specific behavior:

[0603] The user opens the camera app on their smartphone and takes a picture that shows the entire room. Then, they open the dedicated application and tap the "Upload" button to import the image into the application.

[0604] output:

[0605] Image data stored on the smartphone is sent to the server via a dedicated application.

[0606] Step 2:

[0607] Enter and submit relevant information

[0608] input:

[0609] Users fill out forms within the application to enter relevant information such as theme, color, budget, etc.

[0610] Specific behavior:

[0611] Users enter information such as the theme (e.g., vintage style), preferred colors (e.g., brown and gold), and budget (e.g., 150,000 yen) into the application's input form and tap the "Submit" button.

[0612] output:

[0613] Relevant information such as theme, color, budget, etc. is sent to the server and associated with the image data.

[0614] Step 3:

[0615] Acquiring emotional information

[0616] input:

[0617] When users upload images, emotional information is collected using facial recognition technology.

[0618] Specific behavior:

[0619] An emotion engine built into the application analyzes the user's face in real time to detect their emotional state (e.g., relaxed).

[0620] output:

[0621] Emotional state data is generated and transmitted to a server.

[0622] Step 4:

[0623] Data reception and analysis

[0624] input:

[0625] The server receives the image data, related information (theme, color, budget), and emotional information sent by the user.

[0626] Specific behavior:

[0627] The server receives the captured image data, related information, and emotion information and temporarily stores them in a database.

[0628] output:

[0629] Stored image data and related information, as well as emotional information.

[0630] Step 5:

[0631] Image analysis

[0632] input:

[0633] Image data stored on the server.

[0634] Specific behavior:

[0635] The server uses image analysis equipment (e.g., OpenCV or TensorFlow) to analyze the image data and extract spatial features (e.g., dimensions, existing furniture arrangement, interior style).

[0636] output:

[0637] Extracted spatial feature data.

[0638] Step 6:

[0639] Proposal generation

[0640] input:

[0641] Analysis results, related information (theme, color, budget) and sentiment information stored on the server.

[0642] Specific behavior:

[0643] The server sends the analysis results, related information, and emotional information to a generator (e.g., OpenAI's GPT-4) to generate optimal interior design proposals. The generator connects to an interior design database to generate item and layout proposals based on the latest interior trends.

[0644] output:

[0645] Generated interior design proposal data (furniture layout, list of recommended items, and purchase links).

[0646] Step 7:

[0647] Submitting generated proposals

[0648] input:

[0649] Generated interior proposal data.

[0650] Specific behavior:

[0651] The server organizes the generated proposals and sends them to the user's terminal.

[0652] output:

[0653] Interior design suggestions displayed on the user's device.

[0654] Step 8:

[0655] Confirmation and decision on proposal contents

[0656] input:

[0657] The offer received by the user's device.

[0658] Specific behavior:

[0659] Users can review the suggestions within the app, select the one they like, and tap the purchase link.

[0660] output:

[0661] Based on the user's purchasing intent, the user is redirected to a page where they can purchase the suggested furniture and accessories.

[0662] In this way, users can easily and quickly receive interior design suggestions that suit their needs and emotions.

[0663] (Application example 2)

[0664] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0665] In modern interior design, users face the challenge of finding suggestions that reflect their emotions and immediate mood. It is also difficult for users to directly and intuitively understand where suggested items will be placed when selecting products in a physical store. Therefore, it is necessary to provide more personalized interior design suggestions by analyzing users' real-time emotions and states and guiding the location of offered items in a physical store.

[0666] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring an image of the space using an imaging device, means for transmitting the image and related information to the server, means for identifying characteristics of the space using an image analysis device in the server, means for including a generation device that generates optimal furniture and accessory suggestions using the analysis results and related information, means for including an emotion engine that recognizes the user's emotional state and adjusts the suggestions based on the emotion information, and means for transmitting the generated suggestions to a terminal and guiding the location of recommended items. This allows the user to receive interior suggestions tailored to their emotional state in real time and be guided in selecting the locations of specific items in a physical store.

[0667] An "imaging device" is a device that allows a user to capture an image of a space, and includes a smartphone, smart glasses, etc.

[0668] "Related information" refers to information related to interior coordination such as budget, theme, and color provided by the user.

[0669] A "server" is a computer system that receives images and information sent by users and performs analysis and generation.

[0670] An "image analysis device" is a device that analyzes transmitted images and identifies the characteristics, dimensions, existing furniture layout, etc. of a space.

[0671] A "generator" is a device that creates optimal furniture and accessory proposals based on analysis results and related information.

[0672] An "emotion engine" is a system that recognizes emotions from a user's facial expressions, voice, etc. and analyzes that information.

[0673] "Terminal" refers to a device that receives and displays interior design proposals generated by a user, and includes smartphones, smart glasses, etc.

[0674] "Means for guiding the location of recommended items" refers to a function that guides the user to the specific location of recommended products or items in the generated suggestions.

[0675] The "interior design database" is a database that stores information on various interior styles and trends, and is a data source that the generation device references when generating proposals.

[0676] In this invention, a user wears smart glasses and walks around a physical store, photographing the products and space inside the store. The smart glasses capture the scenery and products the user sees in real time and save them as images. The user can also input relevant information such as budget, theme, and color preferences into the smart glasses using voice input. At this time, the smart glasses are equipped with an emotion engine that analyzes the user's emotional state from facial expressions and tone of voice to acquire emotional information.

[0677] The server receives the images, related information, and emotion information sent by the user. The server then uses an image analyzer to identify spatial features (dimensions, existing furniture layout, etc.) from the received images. Software libraries such as OpenCV are used for image analysis, and machine learning models such as TensorFlow are used for emotion analysis.

[0678] The server generates optimal interior design suggestions based on the analysis results, related information, and emotional information. The generator is connected to a global interior design database and can suggest furniture and accessories based on the latest trends. The suggestions are adjusted to suit the user's emotional state.

[0679] The generated suggestions are sent to the user's smart glasses in real time. The smart glasses display guides the user to the location of the recommended items, making it easier for the user to identify them in the store. This allows users to easily coordinate interiors that match their emotions and preferences in a physical store.

[0680] Specific examples

[0681] As the user wears the smart glasses and walks around an interior design shop, the glasses take photos of the furniture and decorations in the store. The user inputs "budget 200,000 yen, theme modern, color preference black and white" by voice, and the emotion engine analyzes the user's real-time emotional state and sends it to the server. The server uses an image analysis device to analyze the dimensions and placement of products in the store, and a generator recommends modern-themed furniture and accessories that match the user's emotional information. The glasses' display shows the location of the recommended item along with the message "This item is perfect for your mood!"

[0682] Prompt statement

[0683] "A user visits an interior design shop and uses the smart glasses to input information such as a budget of 200,000 yen, a modern theme, and a color preference of black and white, all while looking around the shop. The smart glasses also analyze the user's emotions and suggest the optimal interior coordination. The suggestions are displayed on the glasses' display in real time, and the user is guided to the location of items based on the suggestions."

[0684] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0685] Step 1:

[0686] A user wears smart glasses and walks around a physical store. The smart glasses' camera captures images of the store in real time. The user inputs relevant information, such as budget, theme, and color preferences, into the smart glasses using voice input. The smart glasses' camera captures image data, and the voice input device captures the relevant information as text. These data are temporarily stored in the smart glasses' memory.

[0687] Input: In-store images, relevant information such as the user's budget, theme, colors, etc.

[0688] Output: Image data, related text data

[0689] Step 2:

[0690] The emotion engine built into the smart glasses analyzes the user's emotional state in real time from their facial expressions and tone of voice. The emotion engine generates emotional information using an AI model that combines facial expression recognition and voice analysis, and stores this information in the smart glasses. The user's emotional information is also generated as text data.

[0691] Input: User's facial expression, tone of voice

[0692] Output: Emotional information text data

[0693] Step 3:

[0694] The smart glasses transmit the acquired image data, related information, and emotional information to a server via wireless communication (Wi-Fi or Bluetooth). The server receives and stores this data.

[0695] Input: Image data, text data of related information, text data of emotional information

[0696] Output: Data stored on the server

[0697] Step 4:

[0698] The server uses an image analysis device to analyze the received image data and identify the characteristics of the space. Software libraries such as OpenCV are used for image analysis. The analysis results, including the dimensions of the space and the layout of existing furniture, are generated as text data and saved on the server.

[0699] Input: Image data

[0700] Output: Text data of spatial features (dimensions, furniture layout, etc.)

[0701] Step 5:

[0702] The server generates optimal interior design proposals based on the analysis results, related information, and emotional information. The generator connects to a global interior design database, which generates furniture and accessory proposals based on the latest trends. The proposals are adjusted to suit the user's emotional state. The final proposals are generated as text and image data.

[0703] Input: Text data of spatial features, text data of related information, text data of emotional information

[0704] Output: Text data and image data of optimal interior design proposals

[0705] Step 6:

[0706] The server sends the generated interior design proposals to the smart glasses via wireless communication (Wi-Fi or Bluetooth). The smart glasses receive the proposals and store them in their internal memory.

[0707] Input: Text data and image data of optimal interior proposals

[0708] Output: Proposal data stored in smart glasses

[0709] Step 7:

[0710] The smart glasses display the suggested items on a screen. The user is guided to locate the suggested items in a physical store. Augmented reality (AR) technology is used to display the guidance. The location of the suggested item is overlaid on the display, and the user can navigate to that location.

[0711] Input: Proposal data stored in smart glasses

[0712] Output: Suggestions and guidance displayed on the smart glasses display

[0713] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0714] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0715] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0716] [Third embodiment]

[0717] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0718] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0719] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0720] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0721] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0722] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0723] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0724] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0725] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0726] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0727] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0728] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0729] The present invention is a system in which a user acquires an image of a space, transmits the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. A specific embodiment of this system is described below.

[0730] User operation

[0731] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded via a dedicated application. This application provides the user with an interface for entering relevant information such as theme, color, and budget. The user enters this information and presses the "send" button to send the data to the server.

[0732] Server-side processing

[0733] The server receives the image and input information sent by the user. The server then sends the image data to an image analysis device, which analyzes the space's characteristics (e.g., dimensions, existing furniture layout, interior style). The analysis device extracts these characteristics and returns the analysis results to the server.

[0734] The server then sends these analysis results, along with information entered by the user such as theme, color, and budget, to the generator, which then generates optimal interior coordination proposals. The generator connects to interior design databases from around the world and can generate optimal item and layout proposals based on the latest trends.

[0735] Sending generated suggestions to users

[0736] The server then organizes the suggestions returned by the generators and sends them to the user's device, where they can be viewed within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[0737] Specific examples

[0738] For example, if User A has a budget of 100,000 yen, a Nordic theme, and favorite colors of blue and white, the user takes a picture of their living room with their smartphone, enters the relevant information into a dedicated app, and submits it. The server uses an image analysis device to analyze the dimensions of User A's living room and the layout of existing furniture, and the generator then proposes optimal furniture and accessories based on the Nordic theme.

[0739] The proposals are sent to User A's smartphone, and User A can check the proposals within the app. If User A likes the proposed items, he or she can purchase them within the app. Through this series of processes, users can easily realize their ideal interior coordination.

[0740] The above is a specific description of the embodiment of the present invention. This system allows users to receive optimal interior design proposals that meet their individual needs without any effort on their part.

[0741] The processing flow will be explained below.

[0742] Step 1:

[0743] The user takes an image of the room using a camera (smartphone camera).

[0744] Step 2:

[0745] The user launches a dedicated application and uploads the captured images to the app.

[0746] Step 3:

[0747] The user inputs relevant information such as budget, theme, color preferences, etc. through the application interface.

[0748] Step 4:

[0749] The user presses the "send" button to send the image and input information to the server.

[0750] Step 5:

[0751] The server receives the image and input information sent by the user.

[0752] Step 6:

[0753] The server sends the image data to an image analysis device, which analyzes the characteristics of the space (dimensions, existing furniture layout, interior style).

[0754] Step 7:

[0755] The image analyzer analyzes the image and extracts spatial features, and the results are sent back to the server.

[0756] Step 8:

[0757] The server transmits the analysis results and user input information to the generation device, which then generates an optimal interior proposal.

[0758] Step 9:

[0759] The generator generates optimal furniture and accessory proposals based on the analysis results and user input, linking with interior design databases around the world.

[0760] Step 10:

[0761] The server transmits the proposal content returned from the generating device to the user's terminal.

[0762] Step 11:

[0763] The user checks the proposal on their device (smartphone), which includes a furniture layout diagram, a list of recommended items, and a link to purchase.

[0764] Step 12:

[0765] The user selects the suggested item and checks out within the app.

[0766] This allows users to easily create their ideal interior coordination.

[0767] Example 1

[0768] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0769] Conventional interior coordination suggestion systems have the drawback of making it difficult for users to obtain proposals that match their preferences and budget, resulting in a time-consuming and labor-intensive process. Furthermore, obtaining optimal proposals that meet the user's needs requires advanced analysis and generation technology, but such technology is often not fully utilized.

[0770] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0771] In this invention, the server includes means for acquiring images of the space using a photographing device, means for transmitting the images and related information to the server, means for identifying the characteristics of the space using an image analyzing device in the server, means for providing a generating device for generating optimal furniture and accessory proposals using a generative AI model based on the analysis results and related information, and means for transmitting the generated proposals to a terminal. This enables a user to quickly and accurately receive interior coordination proposals based on the user's photographed images and desired conditions.

[0772] "Capture device" refers to a device for capturing images of a space. Examples include smartphones and digital cameras.

[0773] "Server" refers to a computer system that receives, stores, and processes data over a network.

[0774] "Image analysis device" refers to software or hardware for extracting and analyzing features from images. Specific examples include TensorFlow and OpenCV.

[0775] "Generative AI models" refer to artificial intelligence models that generate optimal suggestions based on user criteria. Specific examples include GPT-4.

[0776] "Terminal" refers to a device that receives data from a server and is used as an interface by a user. Examples include smartphones and tablets.

[0777] "Spatial characteristics" refers to information such as the dimensions of the space, existing furniture layout, and interior style obtained through image analysis.

[0778] "Related information" refers to information entered by the user, such as theme, favorite color, budget, etc.

[0779] "Furniture and accessory suggestions" refers to specific interior layout ideas and lists of recommended items generated based on the user's criteria and analysis results.

[0780] "Database" refers to a system for storing and managing interior design data and other related information.

[0781] The present invention is a system in which a user acquires an image of a space, transmits the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. A specific embodiment of this system is described below.

[0782] User operation

[0783] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded via a dedicated application. This application provides the user with an interface for entering relevant information such as the theme, favorite colors, and budget. The user enters this information and presses the "send" button to send the data to the server.

[0784] Server-side processing

[0785] The server receives the image and related information sent by the user. The server then sends the image data to an image analyzer, requesting it to analyze the spatial features (e.g., dimensions, existing furniture layout, interior style). The image analyzer can use software such as TensorFlow or OpenCV. The analyzer extracts these features and returns the analysis results to the server.

[0786] The server then uses a generative AI model (such as GPT-4) to generate optimal interior coordination suggestions based on these analysis results and user information. The generator connects to interior design databases around the world to make suggestions based on the latest trends.

[0787] Sending generated suggestions to users

[0788] The server then organizes the suggestions returned by the generator and sends them to the user's device. The suggestions include a detailed furniture layout plan, a list of recommended items, and links to purchase them. The user can view these suggestions in a dedicated application.

[0789] Specific examples

[0790] For example, the specific operating procedure for User A, who has a budget of 100,000 yen, a Nordic theme, and preferred colors of blue and white, is shown below. User A first takes a picture of their living room with their smartphone and uploads it to a dedicated app. Next, they enter the relevant information into the app's interface and send it to the server. The server uses TensorFlow to analyze the dimensions and existing furniture layout of User A's living room, and uses GPT-4 to generate optimal interior design suggestions. The generated suggestions are sent to User A's smartphone, where User A can view the suggestions within the app.

[0791] Prompt Sentence Examples

[0792] In the case of User A, the generative AI model receives the following prompt: "Please suggest an interior design with a Nordic theme and blue and white colors for a budget of 100,000 yen."

[0793] In this way, by using the system of the present invention, the user can easily achieve the ideal interior coordination.

[0794] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0795] Processing Steps

[0796] Step 1:

[0797] The user takes a picture and sends it to the server through the application.

[0798] Input: User-taken images of your space, along with related information like theme, favorite colors, budget, etc.

[0799] How it works: The user takes a picture of their room using the smartphone camera, uploads it to a dedicated application, and then enters the theme, favorite colors, and budget information through the application interface and presses the send button to send it to the server.

[0800] Step 2:

[0801] A server receives the image and related information.

[0802] Output: Storage of image data and related information sent to the server

[0803] Specific operation: The server receives the image file, theme, favorite color, and budget information sent by the user, and stores them in a database in JSON format. The received data is linked to the user ID.

[0804] Step 3:

[0805] The server sends the image data to an image analysis device, which analyzes the characteristics of the space.

[0806] Input: Image data stored on the server

[0807] Output: Analysis results such as space dimensions, existing furniture arrangement, and interior style

[0808] How it works: The server sends the stored images to an image analyzer (e.g., TensorFlow or OpenCV), which extracts feature information from the images, such as the dimensions of the space, the existing furniture layout, and the interior style, and returns the analysis results to the server.

[0809] Step 4:

[0810] The server transmits the analysis results and user information to the generation device.

[0811] Input: Analysis results and related information entered by the user (theme, color, budget)

[0812] Output: Interior coordination suggestions generated by the generative AI model

[0813] Specific operation: The server combines the analysis results with the user's relevant information to create and send a prompt to the generative AI model (e.g., GPT-4). An example prompt is "Please suggest an interior coordination with a Nordic theme and blue and white colors for a budget of 100,000 yen." The generative AI model then generates the optimal interior coordination proposal and sends it back to the server.

[0814] Step 5:

[0815] The server organizes the generated results and sends them to the user terminal.

[0816] Input: Coordinate proposals from the generator

[0817] Output: Sending the organized proposals to the user's device

[0818] Specific operation: The server organizes the proposals received from the generator, formats them appropriately, and sends them to the user's device. The proposals include a detailed furniture layout plan, a list of recommended items, and links to purchase them.

[0819] Step 6:

[0820] The user reviews the suggestions within the application and purchases the items if necessary.

[0821] Input: Interior coordination proposal sent from the server

[0822] What it does: The user opens the app and sees the suggestions sent from the server, including a detailed furniture layout and a list of recommended items, and can purchase the items they like right within the app.

[0823] The above is the specific processing flow of the program for this system, as well as the input, output, and specific operation at each step.

[0824] (Application example 1)

[0825] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0826] Conventional interior coordination systems only allow users to check the proposed interior designs online, making it difficult for them to actually use and check the products. As a result, users are unable to check the actual appearance and texture of the products before purchasing them, which has the problem of not being able to fully stimulate users' purchasing motivation.

[0827] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0828] In this invention, the server includes means for capturing an image of the space using a photographing device, means for transmitting the image and related information to the server, means for identifying characteristics of the space using an image analyzing device in the server, means for providing a generating device for generating optimal furniture and accessory proposals using the analysis results and related information, means for transmitting the generated proposals to a terminal, and means for supporting a user in using furniture and interior products in a physical store to confirm and purchase the proposals. This allows the user to see, touch, and confirm the actual products in the physical store, enabling them to concretely experience the proposed interior coordination.

[0829] A "photography device" is a device used by a user to capture an image of a space, and typically refers to a smartphone or digital camera.

[0830] "Means for transmitting images and related information to the server" refers to a method for transmitting image data acquired by the photographing device and information such as the theme, budget, and color entered by the user to the server, and is exemplified by a dedicated application.

[0831] An "image analysis device" is a device or software that analyzes image data sent to a server and identifies spatial characteristics, and can use image recognition technology, for example.

[0832] A "generator" is a device or software that uses analysis results and related information to generate optimal furniture and accessory suggestions, and can incorporate the latest interior design trends in conjunction with a database.

[0833] "Means for transmitting the generated proposal to the terminal" refers to a method for transmitting the interior coordination proposal generated by the generating device to the user's terminal, and corresponds to data transmission via the Internet.

[0834] "Means to assist users in checking and purchasing suggestions using furniture and interior products in a physical store" refers to methods for supporting users in checking and purchasing provided interior suggestions using actual furniture and interior products in a physical store, and includes in-store displays and assistance from sales staff.

[0835] System Overview

[0836] The interior coordination suggestion system of this invention is based on the user capturing an image of a space using a camera and sending it along with related information to a server. The server then analyzes the received image using an image analysis device to identify the characteristics of the space. Next, a generation device generates optimal furniture and accessory suggestions based on the user's theme, budget, and color information. The generated suggestions are sent to the user's device, where the user can view and purchase the suggested items in a physical store.

[0837] Hardware and software used

[0838] 1. Imaging equipment

[0839] Images of the space are captured using a smartphone or digital camera.

[0840] 2. Server

[0841] The server acts as an API server using Flask and receives images and related information sent by users.

[0842] 3. Image analysis equipment

[0843] Image analysis software such as OpenCV and TensorFlow is used to detect spatial features (dimensions, furniture arrangement, etc.).

[0844] 4. Generator

[0845] The generator works with an internal database or an external interior design database and uses an AI model (e.g., a Transformer-based generative model) to generate optimal coordination suggestions.

[0846] 5. Terminal

[0847] A smartphone or tablet for users to view the suggestions.

[0848] Processing flow

[0849] After receiving the image and related information from the user, the server first analyzes the spatial features using an image analyzer, primarily using OpenCV to detect feature points and edges in the image, and TensorFlow to recognize objects such as furniture.

[0850] Next, the generator generates an interior coordination proposal based on the space's characteristics and the theme, budget, and color information provided by the user. The generator works in conjunction with an interior design database to make proposals that incorporate the latest designs.

[0851] The generated coordination proposals are sent to the user's device via the server, where the user can check and purchase the proposed furniture and accessories in the physical store.

[0852] Specific examples

[0853] User A takes a picture of the living room using his smartphone, and then enters information such as the theme "Scandinavian style," budget "100,000 yen," and favorite colors "blue and white" through the application and sends it.

[0854] The server uses an image analyzer to analyze the dimensions of the living room and the layout of the existing furniture, and a generator generates optimal furniture and accessory suggestions based on a Nordic theme. For example, the suggestions might look like this:

[0855] Prompt Sentence Examples

[0856] User theme: Nordic style

[0857] Budget: 100,000 yen

[0858] Favorite colors: Blue and white

[0859] The generated suggestions are sent to User A's smartphone, and User A can actually check and touch the suggested items in a physical store. User A can then purchase the products they like on the spot.

[0860] Through this process, users can create their ideal interior coordination both online and in-store.

[0861] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0862] Step 1:

[0863] The user takes a picture of their room using their smartphone. They open a dedicated application on the device, enter relevant information such as the theme (e.g., Scandinavian style), budget (e.g., 100,000 yen), and preferred colors (e.g., blue and white), and send the image and text information to the server. In this step, the input data is the captured image and text information, and the output data is generated to be sent to the server.

[0864] Step 2:

[0865] The server receives the images and related information sent by the user. On the server, an image analysis device receives the images and uses image recognition technology (e.g., OpenCV, TensorFlow) to identify spatial features (dimensions, furniture placement, etc.). In this process, the image stored on the server is used as input data, and analyzed spatial feature information is generated as output.

[0866] Step 3:

[0867] The server sends the analyzed space characteristics information and the theme, budget, and preferred color information received from the user to the generator. The generator receives this information as input data and works with an interior design database to generate optimal interior coordination proposals. Here, coordination proposals are generated using an AI model (e.g., a Transformer-based generative model) based on existing design data in the database. The generated proposal data is generated as output.

[0868] Step 4:

[0869] The server organizes the proposals returned from the generation device and sends them to the user's terminal. In this step, the generated proposal data is used as input data, and data to be sent to the user's terminal is prepared as output.

[0870] Step 5:

[0871] The user checks the proposed interior coordination on the device. The proposals are displayed through a dedicated application, and the user can actually check and touch the proposed items in a physical store. At this stage, the input is the proposal data from the server, and the output is the user's visual and tactile confirmation and purchase of the products.

[0872] Step 6:

[0873] If the user likes an item, they can proceed with the purchase process at a physical store. Support is provided for purchasing the suggested interior items on the spot, and the purchase information is processed through a dedicated application. This process takes the user's purchase intention and payment information as input data, and generates purchase completion data as output.

[0874] The above steps implement a series of operations, allowing users to experience the interior coordination in detail and then check and purchase in a physical store to realize their ideal space.

[0875] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0876] The present invention is a system in which a user acquires an image of a space, sends the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. Furthermore, by combining the present invention with an emotion engine that recognizes the user's emotions, the system can make proposals that take into account the user's emotional state. Specific embodiments are described below.

[0877] User operation

[0878] First, users take pictures of their rooms using a camera such as a smartphone. The images are then uploaded via a dedicated application. This application provides users with an interface for entering relevant information such as theme, color, and budget. The application also incorporates an emotion engine that recognizes the user's emotions, and emotional information is acquired simultaneously when the user uploads the image.

[0879] Server-side processing

[0880] The server receives the image, input information, and emotion information sent by the user. The server then sends the image data to an image analysis device and requests it to analyze the spatial features (e.g., dimensions, existing furniture layout, interior style). The image analysis device extracts the features and returns the analysis results to the server.

[0881] The analysis results and related information are sent to the generator along with the emotion information. The generator then uses this information to generate optimal interior design proposals. The generator then connects with interior design databases around the world to generate item and layout proposals based on the latest trends. Furthermore, the generator takes the user's emotional state into account and makes proposals that correspond to their emotions.

[0882] Sending generated suggestions to users

[0883] The server then organizes the suggestions returned by the generators and sends them to the user's device, where they can be viewed within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[0884] Specific examples

[0885] For example, if User B has a budget of 150,000 yen, a vintage theme, and preferred colors of brown and gold, the user can take a picture of their living room with their smartphone, enter the relevant information into a dedicated app, and send it. At this time, the emotion engine also recognizes User B's real-time emotional state and sends it to the server. The server then uses an image analysis device to analyze the dimensions of User B's living room and the layout of existing furniture, and the generator then suggests optimal furniture and accessories based on the vintage theme.

[0886] The suggestions are sent to User B's smartphone, and User B checks the suggestions within the app. If the suggested items match their emotions and they like them, User B can purchase them within the app. Through this process, users can easily create their ideal interior coordination that is in line with their emotions.

[0887] The above is a specific description of the embodiment of the present invention. This system allows users to receive optimal interior design suggestions that meet their individual needs and emotions without any effort on their part.

[0888] The processing flow will be explained below.

[0889] Step 1:

[0890] The user takes an image of the room using a camera (smartphone camera).

[0891] Step 2:

[0892] The user launches a dedicated application and uploads the captured images to the app.

[0893] Step 3:

[0894] Users input relevant information such as budget, theme, color preferences, etc. through the application interface, and the emotion engine automatically captures emotional information from the user's facial expressions and voice.

[0895] Step 4:

[0896] The user presses the "send" button to send the image, input information, and emotion information to the server.

[0897] Step 5:

[0898] The server receives the image, input information, and emotion information sent by the user.

[0899] Step 6:

[0900] The server sends the received image data to an image analysis device, which analyzes the characteristics of the space (dimensions, arrangement of existing furniture, interior style, etc.).

[0901] Step 7:

[0902] The image analysis device performs the analysis and returns the results to the server.

[0903] Step 8:

[0904] The server transmits the analysis results, information about the theme, budget, and color preferences entered by the user, and emotional information to the generation device.

[0905] Step 9:

[0906] The generator generates optimal furniture and accessory recommendations based on the analysis results and user input. The generator also takes into account the user's emotional information to generate recommendations that are appropriate to their emotions. The generator also connects to a global interior design database, enabling it to provide recommendations based on the latest trends.

[0907] Step 10:

[0908] The generator sends the generated proposal back to the server.

[0909] Step 11:

[0910] The server organizes the proposals and sends them to the user's device.

[0911] Step 12:

[0912] The user checks the proposal on their device (smartphone), which includes a new furniture layout, a list of recommended items, and a link to purchase.

[0913] Step 13:

[0914] The user reviews the suggestions and selects the items they like. The user completes the purchase within the app.

[0915] This allows users to receive basic interior coordination suggestions as well as emotional suggestions. For example, if a user wants to relax, the app will suggest furniture and accessories in calming colors.

[0916] Example 2

[0917] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0918] Conventional interior design suggestion systems have difficulty making suggestions that fully consider the user's individual needs and emotional state. Another problem is the lack of technology that can accurately grasp the characteristics of a space based on image analysis and generate optimal interior design suggestions based on that information. This means that users have to spend a lot of time selecting the interior design themselves, and are unable to obtain satisfactory suggestions.

[0919] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0920] In this invention, the server includes means for capturing images of the space using a photographing device, means for transmitting the images and related information to the server, means for identifying characteristics of the space using an image analyzing device, means for providing a generating device for generating optimal furniture and accessory proposals, means for transmitting the generated proposals to the terminal, and means for acquiring the user's emotional information using an emotion engine and reflecting the information in the generated proposals. This enables the user to easily receive optimal interior proposals that match their individual needs and emotional state based on the captured images, related information, and emotional information.

[0921] A "photography device" is a device that a user uses to capture an image of a space, and typically refers to a smartphone or digital camera equipped with a camera function.

[0922] "Means for transmitting images and related information to a server" refers to a function for transmitting images acquired by a user and information such as theme, budget, and preferred colors to a server via the Internet.

[0923] An "image analysis device" refers to software or hardware that analyzes received image data and extracts spatial characteristics (dimensions, furniture arrangement, interior style, etc.).

[0924] A "generator" is a system that generates optimal furniture and accessory suggestions based on image analysis results and related information, and is typically implemented using a generative AI model.

[0925] The "means for transmitting the generated proposal to the terminal" refers to a communication function for transmitting the interior proposal generated by the generation device to the user's terminal.

[0926] An "emotion engine" is a system for analyzing and acquiring the user's emotional state, and for example, recognizes the user's emotions using facial expression recognition technology.

[0927] The "Interior Design Database" is a database that accumulates data on interior design from around the world and provides information on the latest trends and items.

[0928] MODE FOR CARRYING OUT THE INVENTION

[0929] The present invention is a system in which a user acquires an image of a space, sends the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. Furthermore, by combining the present invention with an emotion engine that recognizes the user's emotions, it is possible to make proposals that take into account the user's emotional state. Specific embodiments are described below.

[0930] User operation

[0931] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded to a server via a dedicated application. This application provides the user with an interface for entering relevant information such as theme, color, and budget. The application also incorporates an emotion engine, which simultaneously acquires emotional information when the user uploads the image. The emotion engine uses facial expression recognition technology to recognize the user's emotions in real time.

[0932] Server-side processing

[0933] The server receives the image, related information, and emotional information sent by the user. The server then sends the image data to an image analyzer (e.g., an image analysis library such as OpenCV or TensorFlow) to analyze the spatial features (e.g., dimensions, existing furniture arrangement, interior style, etc.). The image analyzer then uses a deep learning model to extract these features from the image.

[0934] The server then sends the analysis results, related information, and emotional information to a generator (such as OpenAI's GPT-4, an example of a generative AI model) to generate optimal interior design suggestions. The generator connects with interior design databases around the world to generate item and layout suggestions based on the latest trends. It also makes suggestions that take the user's emotional state into account.

[0935] Sending generated suggestions to users

[0936] The server then compiles the suggestions returned by the generators and sends them to the user's device, where they can view them within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[0937] Specific examples

[0938] For example, if User B sets a budget of 150,000 yen, a vintage theme, and favorite colors of brown and gold, User B takes a picture of their living room with their smartphone, enters the image and related information into a dedicated app, and sends it. At this time, the emotion engine built into the app recognizes User B's emotional state in real time and sends it to the server.

[0939] The server uses OpenCV to analyze the dimensions and existing furniture layout of User B's living room. The analysis results are sent back to the server, which then uses this information to generate optimal furniture and accessory recommendations based on a vintage theme using tools such as OpenAI's GPT-4. Once the recommendations are sent to User B's smartphone, User B can review the recommendations—for example, a vintage-style sofa, lamp, or carpet—in the app, and if they like them, they can simply tap the purchase link to purchase them.

[0940] Prompt Sentence Examples

[0941] "I'd like some suggestions for a vintage-inspired interior. My budget is ¥150,000, my favorite colors are brown and gold, and my current emotional state is relaxed. The image of my living room includes my current furniture arrangement. Please suggest the best arrangement and items."

[0942] The above is a specific embodiment of the present invention. This system allows users to easily create an ideal interior coordination that is in line with their emotions.

[0943] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0944] Step 1:

[0945] Taking photos and uploading images

[0946] input:

[0947] The user takes a picture of their room using the smartphone camera.

[0948] Specific behavior:

[0949] The user opens the camera app on their smartphone and takes a picture that shows the entire room. Then, they open the dedicated application and tap the "Upload" button to import the image into the application.

[0950] output:

[0951] Image data stored on the smartphone is sent to the server via a dedicated application.

[0952] Step 2:

[0953] Enter and submit relevant information

[0954] input:

[0955] Users fill out forms within the application to enter relevant information such as theme, color, budget, etc.

[0956] Specific behavior:

[0957] Users enter information such as the theme (e.g., vintage style), preferred colors (e.g., brown and gold), and budget (e.g., 150,000 yen) into the application's input form and tap the "Submit" button.

[0958] output:

[0959] Relevant information such as theme, color, budget, etc. is sent to the server and associated with the image data.

[0960] Step 3:

[0961] Acquiring emotional information

[0962] input:

[0963] When users upload images, emotional information is collected using facial recognition technology.

[0964] Specific behavior:

[0965] An emotion engine built into the application analyzes the user's face in real time to detect their emotional state (e.g., relaxed).

[0966] output:

[0967] Emotional state data is generated and transmitted to a server.

[0968] Step 4:

[0969] Data reception and analysis

[0970] input:

[0971] The server receives the image data, related information (theme, color, budget), and emotional information sent by the user.

[0972] Specific behavior:

[0973] The server receives the captured image data, related information, and emotion information and temporarily stores them in a database.

[0974] output:

[0975] Stored image data and related information, as well as emotional information.

[0976] Step 5:

[0977] Image analysis

[0978] input:

[0979] Image data stored on the server.

[0980] Specific behavior:

[0981] The server uses image analysis equipment (e.g., OpenCV or TensorFlow) to analyze the image data and extract spatial features (e.g., dimensions, existing furniture arrangement, interior style).

[0982] output:

[0983] Extracted spatial feature data.

[0984] Step 6:

[0985] Proposal generation

[0986] input:

[0987] Analysis results, related information (theme, color, budget) and sentiment information stored on the server.

[0988] Specific behavior:

[0989] The server sends the analysis results, related information, and emotional information to a generator (e.g., OpenAI's GPT-4) to generate optimal interior design proposals. The generator connects to an interior design database to generate item and layout proposals based on the latest interior trends.

[0990] output:

[0991] Generated interior design proposal data (furniture layout, list of recommended items, and purchase links).

[0992] Step 7:

[0993] Submitting generated proposals

[0994] input:

[0995] Generated interior proposal data.

[0996] Specific behavior:

[0997] The server organizes the generated proposals and sends them to the user's terminal.

[0998] output:

[0999] Interior design suggestions displayed on the user's device.

[1000] Step 8:

[1001] Confirmation and decision on proposal contents

[1002] input:

[1003] The offer received by the user's device.

[1004] Specific behavior:

[1005] Users can review the suggestions within the app, select the one they like, and tap the purchase link.

[1006] output:

[1007] Based on the user's purchasing intent, the user is redirected to a page where they can purchase the suggested furniture and accessories.

[1008] In this way, users can easily and quickly receive interior design suggestions that suit their needs and emotions.

[1009] (Application example 2)

[1010] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1011] In modern interior design, users face the challenge of finding suggestions that reflect their emotions and immediate mood. It is also difficult for users to directly and intuitively understand where suggested items will be placed when selecting products in a physical store. Therefore, it is necessary to provide more personalized interior design suggestions by analyzing users' real-time emotions and states and guiding the location of offered items in a physical store.

[1012] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring an image of the space using an imaging device, means for transmitting the image and related information to the server, means for identifying characteristics of the space using an image analysis device in the server, means for including a generation device that generates optimal furniture and accessory suggestions using the analysis results and related information, means for including an emotion engine that recognizes the user's emotional state and adjusts the suggestions based on the emotion information, and means for transmitting the generated suggestions to a terminal and guiding the location of recommended items. This allows the user to receive interior suggestions tailored to their emotional state in real time and be guided in selecting the locations of specific items in a physical store.

[1013] An "imaging device" is a device that allows a user to capture an image of a space, and includes a smartphone, smart glasses, etc.

[1014] "Related information" refers to information related to interior coordination such as budget, theme, and color provided by the user.

[1015] A "server" is a computer system that receives images and information sent by users and performs analysis and generation.

[1016] An "image analysis device" is a device that analyzes transmitted images and identifies the characteristics, dimensions, existing furniture layout, etc. of a space.

[1017] A "generator" is a device that creates optimal furniture and accessory proposals based on analysis results and related information.

[1018] An "emotion engine" is a system that recognizes emotions from a user's facial expressions, voice, etc. and analyzes that information.

[1019] "Terminal" refers to a device that receives and displays interior design proposals generated by a user, and includes smartphones, smart glasses, etc.

[1020] "Means for guiding the location of recommended items" refers to a function that guides the user to the specific location of recommended products or items in the generated suggestions.

[1021] The "interior design database" is a database that stores information on various interior styles and trends, and is a data source that the generation device references when generating proposals.

[1022] In this invention, a user wears smart glasses and walks around a physical store, photographing the products and space inside the store. The smart glasses capture the scenery and products the user sees in real time and save them as images. The user can also input relevant information such as budget, theme, and color preferences into the smart glasses using voice input. At this time, the smart glasses are equipped with an emotion engine that analyzes the user's emotional state from facial expressions and tone of voice to acquire emotional information.

[1023] The server receives the images, related information, and emotion information sent by the user. The server then uses an image analyzer to identify spatial features (dimensions, existing furniture layout, etc.) from the received images. Software libraries such as OpenCV are used for image analysis, and machine learning models such as TensorFlow are used for emotion analysis.

[1024] The server generates optimal interior design suggestions based on the analysis results, related information, and emotional information. The generator is connected to a global interior design database and can suggest furniture and accessories based on the latest trends. The suggestions are adjusted to suit the user's emotional state.

[1025] The generated suggestions are sent to the user's smart glasses in real time. The smart glasses display guides the user to the location of the recommended items, making it easier for the user to identify them in the store. This allows users to easily coordinate interiors that match their emotions and preferences in a physical store.

[1026] Specific examples

[1027] As the user wears the smart glasses and walks around an interior design shop, the glasses take photos of the furniture and decorations in the store. The user inputs "budget 200,000 yen, theme modern, color preference black and white" by voice, and the emotion engine analyzes the user's real-time emotional state and sends it to the server. The server uses an image analysis device to analyze the dimensions and placement of products in the store, and a generator recommends modern-themed furniture and accessories that match the user's emotional information. The glasses' display shows the location of the recommended item along with the message "This item is perfect for your mood!"

[1028] Prompt statement

[1029] "A user visits an interior design shop and uses the smart glasses to input information such as a budget of 200,000 yen, a modern theme, and a color preference of black and white, all while looking around the shop. The smart glasses also analyze the user's emotions and suggest the optimal interior coordination. The suggestions are displayed on the glasses' display in real time, and the user is guided to the location of items based on the suggestions."

[1030] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1031] Step 1:

[1032] A user wears smart glasses and walks around a physical store. The smart glasses' camera captures images of the store in real time. The user inputs relevant information, such as budget, theme, and color preferences, into the smart glasses using voice input. The smart glasses' camera captures image data, and the voice input device captures the relevant information as text. These data are temporarily stored in the smart glasses' memory.

[1033] Input: In-store images, relevant information such as the user's budget, theme, colors, etc.

[1034] Output: Image data, related text data

[1035] Step 2:

[1036] The emotion engine built into the smart glasses analyzes the user's emotional state in real time from their facial expressions and tone of voice. The emotion engine generates emotional information using an AI model that combines facial expression recognition and voice analysis, and stores this information in the smart glasses. The user's emotional information is also generated as text data.

[1037] Input: User's facial expression, tone of voice

[1038] Output: Emotional information text data

[1039] Step 3:

[1040] The smart glasses transmit the acquired image data, related information, and emotional information to a server via wireless communication (Wi-Fi or Bluetooth). The server receives and stores this data.

[1041] Input: Image data, text data of related information, text data of emotional information

[1042] Output: Data stored on the server

[1043] Step 4:

[1044] The server uses an image analysis device to analyze the received image data and identify the characteristics of the space. Software libraries such as OpenCV are used for image analysis. The analysis results, including the dimensions of the space and the layout of existing furniture, are generated as text data and saved on the server.

[1045] Input: Image data

[1046] Output: Text data of spatial features (dimensions, furniture layout, etc.)

[1047] Step 5:

[1048] The server generates optimal interior design proposals based on the analysis results, related information, and emotional information. The generator connects to a global interior design database, which generates furniture and accessory proposals based on the latest trends. The proposals are adjusted to suit the user's emotional state. The final proposals are generated as text and image data.

[1049] Input: Text data of spatial features, text data of related information, text data of emotional information

[1050] Output: Text data and image data of optimal interior design proposals

[1051] Step 6:

[1052] The server sends the generated interior design proposals to the smart glasses via wireless communication (Wi-Fi or Bluetooth). The smart glasses receive the proposals and store them in their internal memory.

[1053] Input: Text data and image data of optimal interior proposals

[1054] Output: Proposal data stored in smart glasses

[1055] Step 7:

[1056] The smart glasses display the suggested items on a screen. The user is guided to locate the suggested items in a physical store. Augmented reality (AR) technology is used to display the guidance. The location of the suggested item is overlaid on the display, and the user can navigate to that location.

[1057] Input: Proposal data stored in smart glasses

[1058] Output: Suggestions and guidance displayed on the smart glasses display

[1059] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1060] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1061] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1062] [Fourth embodiment]

[1063] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1064] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1065] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1066] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1067] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1068] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1069] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1070] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1071] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1072] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1073] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1074] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1075] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1076] The present invention is a system in which a user acquires an image of a space, transmits the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. A specific embodiment of this system is described below.

[1077] User operation

[1078] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded via a dedicated application. This application provides the user with an interface for entering relevant information such as theme, color, and budget. The user enters this information and presses the "send" button to send the data to the server.

[1079] Server-side processing

[1080] The server receives the image and input information sent by the user. The server then sends the image data to an image analysis device, which analyzes the space's characteristics (e.g., dimensions, existing furniture layout, interior style). The analysis device extracts these characteristics and returns the analysis results to the server.

[1081] The server then sends these analysis results, along with information entered by the user such as theme, color, and budget, to the generator, which then generates optimal interior coordination proposals. The generator connects to interior design databases from around the world and can generate optimal item and layout proposals based on the latest trends.

[1082] Sending generated suggestions to users

[1083] The server then organizes the suggestions returned by the generators and sends them to the user's device, where they can be viewed within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[1084] Specific examples

[1085] For example, if User A has a budget of 100,000 yen, a Nordic theme, and favorite colors of blue and white, the user takes a picture of their living room with their smartphone, enters the relevant information into a dedicated app, and submits it. The server uses an image analysis device to analyze the dimensions of User A's living room and the layout of existing furniture, and the generator then proposes optimal furniture and accessories based on the Nordic theme.

[1086] The proposals are sent to User A's smartphone, and User A can check the proposals within the app. If User A likes the proposed items, he or she can purchase them within the app. Through this series of processes, users can easily realize their ideal interior coordination.

[1087] The above is a specific description of the embodiment of the present invention. This system allows users to receive optimal interior design proposals that meet their individual needs without any effort on their part.

[1088] The processing flow will be explained below.

[1089] Step 1:

[1090] The user takes an image of the room using a camera (smartphone camera).

[1091] Step 2:

[1092] The user launches a dedicated application and uploads the captured images to the app.

[1093] Step 3:

[1094] The user inputs relevant information such as budget, theme, color preferences, etc. through the application interface.

[1095] Step 4:

[1096] The user presses the "send" button to send the image and input information to the server.

[1097] Step 5:

[1098] The server receives the image and input information sent by the user.

[1099] Step 6:

[1100] The server sends the image data to an image analysis device, which analyzes the characteristics of the space (dimensions, existing furniture layout, interior style).

[1101] Step 7:

[1102] The image analyzer analyzes the image and extracts spatial features, and the results are sent back to the server.

[1103] Step 8:

[1104] The server transmits the analysis results and user input information to the generation device, which then generates an optimal interior proposal.

[1105] Step 9:

[1106] The generator generates optimal furniture and accessory proposals based on the analysis results and user input, linking with interior design databases around the world.

[1107] Step 10:

[1108] The server transmits the proposal content returned from the generating device to the user's terminal.

[1109] Step 11:

[1110] The user checks the proposal on their device (smartphone), which includes a furniture layout diagram, a list of recommended items, and a link to purchase.

[1111] Step 12:

[1112] The user selects the suggested item and checks out within the app.

[1113] This allows users to easily create their ideal interior coordination.

[1114] Example 1

[1115] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1116] Conventional interior coordination suggestion systems have the drawback of making it difficult for users to obtain proposals that match their preferences and budget, resulting in a time-consuming and labor-intensive process. Furthermore, obtaining optimal proposals that meet the user's needs requires advanced analysis and generation technology, but such technology is often not fully utilized.

[1117] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1118] In this invention, the server includes means for acquiring images of the space using a photographing device, means for transmitting the images and related information to the server, means for identifying the characteristics of the space using an image analyzing device in the server, means for providing a generating device for generating optimal furniture and accessory proposals using a generative AI model based on the analysis results and related information, and means for transmitting the generated proposals to a terminal. This enables a user to quickly and accurately receive interior coordination proposals based on the user's photographed images and desired conditions.

[1119] "Capture device" refers to a device for capturing images of a space. Examples include smartphones and digital cameras.

[1120] "Server" refers to a computer system that receives, stores, and processes data over a network.

[1121] "Image analysis device" refers to software or hardware for extracting and analyzing features from images. Specific examples include TensorFlow and OpenCV.

[1122] "Generative AI models" refer to artificial intelligence models that generate optimal suggestions based on user criteria. Specific examples include GPT-4.

[1123] "Terminal" refers to a device that receives data from a server and is used as an interface by a user. Examples include smartphones and tablets.

[1124] "Spatial characteristics" refers to information such as the dimensions of the space, existing furniture layout, and interior style obtained through image analysis.

[1125] "Related information" refers to information entered by the user, such as theme, favorite color, budget, etc.

[1126] "Furniture and accessory suggestions" refers to specific interior layout ideas and lists of recommended items generated based on the user's criteria and analysis results.

[1127] "Database" refers to a system for storing and managing interior design data and other related information.

[1128] The present invention is a system in which a user acquires an image of a space, transmits the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. A specific embodiment of this system is described below.

[1129] User operation

[1130] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded via a dedicated application. This application provides the user with an interface for entering relevant information such as the theme, favorite colors, and budget. The user enters this information and presses the "send" button to send the data to the server.

[1131] Server-side processing

[1132] The server receives the image and related information sent by the user. The server then sends the image data to an image analyzer, requesting it to analyze the spatial features (e.g., dimensions, existing furniture layout, interior style). The image analyzer can use software such as TensorFlow or OpenCV. The analyzer extracts these features and returns the analysis results to the server.

[1133] The server then uses a generative AI model (such as GPT-4) to generate optimal interior coordination suggestions based on these analysis results and user information. The generator connects to interior design databases around the world to make suggestions based on the latest trends.

[1134] Sending generated suggestions to users

[1135] The server then organizes the suggestions returned by the generator and sends them to the user's device. The suggestions include a detailed furniture layout plan, a list of recommended items, and links to purchase them. The user can view these suggestions in a dedicated application.

[1136] Specific examples

[1137] For example, the specific operating procedure for User A, who has a budget of 100,000 yen, a Nordic theme, and preferred colors of blue and white, is shown below. User A first takes a picture of their living room with their smartphone and uploads it to a dedicated app. Next, they enter the relevant information into the app's interface and send it to the server. The server uses TensorFlow to analyze the dimensions and existing furniture layout of User A's living room, and uses GPT-4 to generate optimal interior design suggestions. The generated suggestions are sent to User A's smartphone, where User A can view the suggestions within the app.

[1138] Prompt Sentence Examples

[1139] In the case of User A, the generative AI model receives the following prompt: "Please suggest an interior design with a Nordic theme and blue and white colors for a budget of 100,000 yen."

[1140] In this way, by using the system of the present invention, the user can easily achieve the ideal interior coordination.

[1141] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1142] Processing Steps

[1143] Step 1:

[1144] The user takes a picture and sends it to the server through the application.

[1145] Input: User-taken images of your space, along with related information like theme, favorite colors, budget, etc.

[1146] How it works: The user takes a picture of their room using the smartphone camera, uploads it to a dedicated application, and then enters the theme, favorite colors, and budget information through the application interface and presses the send button to send it to the server.

[1147] Step 2:

[1148] A server receives the image and related information.

[1149] Output: Storage of image data and related information sent to the server

[1150] Specific operation: The server receives the image file, theme, favorite color, and budget information sent by the user, and stores them in a database in JSON format. The received data is linked to the user ID.

[1151] Step 3:

[1152] The server sends the image data to an image analysis device, which analyzes the characteristics of the space.

[1153] Input: Image data stored on the server

[1154] Output: Analysis results such as space dimensions, existing furniture arrangement, and interior style

[1155] How it works: The server sends the stored images to an image analyzer (e.g., TensorFlow or OpenCV), which extracts feature information from the images, such as the dimensions of the space, the existing furniture layout, and the interior style, and returns the analysis results to the server.

[1156] Step 4:

[1157] The server transmits the analysis results and user information to the generation device.

[1158] Input: Analysis results and related information entered by the user (theme, color, budget)

[1159] Output: Interior coordination suggestions generated by the generative AI model

[1160] Specific operation: The server combines the analysis results with the user's relevant information to create and send a prompt to the generative AI model (e.g., GPT-4). An example prompt is "Please suggest an interior coordination with a Nordic theme and blue and white colors for a budget of 100,000 yen." The generative AI model then generates the optimal interior coordination proposal and sends it back to the server.

[1161] Step 5:

[1162] The server organizes the generated results and sends them to the user terminal.

[1163] Input: Coordinate proposals from the generator

[1164] Output: Sending the organized proposals to the user's device

[1165] Specific operation: The server organizes the proposals received from the generator, formats them appropriately, and sends them to the user's device. The proposals include a detailed furniture layout plan, a list of recommended items, and links to purchase them.

[1166] Step 6:

[1167] The user reviews the suggestions within the application and purchases the items if necessary.

[1168] Input: Interior coordination proposal sent from the server

[1169] What it does: The user opens the app and sees the suggestions sent from the server, including a detailed furniture layout and a list of recommended items, and can purchase the items they like right within the app.

[1170] The above is the specific processing flow of the program for this system, as well as the input, output, and specific operation at each step.

[1171] (Application example 1)

[1172] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1173] Conventional interior coordination systems only allow users to check the proposed interior designs online, making it difficult for them to actually use and check the products. As a result, users are unable to check the actual appearance and texture of the products before purchasing them, which has the problem of not being able to fully stimulate users' purchasing motivation.

[1174] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1175] In this invention, the server includes means for capturing an image of the space using a photographing device, means for transmitting the image and related information to the server, means for identifying characteristics of the space using an image analyzing device in the server, means for providing a generating device for generating optimal furniture and accessory proposals using the analysis results and related information, means for transmitting the generated proposals to a terminal, and means for supporting a user in using furniture and interior products in a physical store to confirm and purchase the proposals. This allows the user to see, touch, and confirm the actual products in the physical store, enabling them to concretely experience the proposed interior coordination.

[1176] A "photography device" is a device used by a user to capture an image of a space, and typically refers to a smartphone or digital camera.

[1177] "Means for transmitting images and related information to the server" refers to a method for transmitting image data acquired by the photographing device and information such as the theme, budget, and color entered by the user to the server, and is exemplified by a dedicated application.

[1178] An "image analysis device" is a device or software that analyzes image data sent to a server and identifies spatial characteristics, and can use image recognition technology, for example.

[1179] A "generator" is a device or software that uses analysis results and related information to generate optimal furniture and accessory suggestions, and can incorporate the latest interior design trends in conjunction with a database.

[1180] "Means for transmitting the generated proposal to the terminal" refers to a method for transmitting the interior coordination proposal generated by the generating device to the user's terminal, and corresponds to data transmission via the Internet.

[1181] "Means to assist users in checking and purchasing suggestions using furniture and interior products in a physical store" refers to methods for supporting users in checking and purchasing provided interior suggestions using actual furniture and interior products in a physical store, and includes in-store displays and assistance from sales staff.

[1182] System Overview

[1183] The interior coordination suggestion system of this invention is based on the user capturing an image of a space using a camera and sending it along with related information to a server. The server then analyzes the received image using an image analysis device to identify the characteristics of the space. Next, a generation device generates optimal furniture and accessory suggestions based on the user's theme, budget, and color information. The generated suggestions are sent to the user's device, where the user can view and purchase the suggested items in a physical store.

[1184] Hardware and software used

[1185] 1. Imaging equipment

[1186] Images of the space are captured using a smartphone or digital camera.

[1187] 2. Server

[1188] The server acts as an API server using Flask and receives images and related information sent by users.

[1189] 3. Image analysis equipment

[1190] Image analysis software such as OpenCV and TensorFlow is used to detect spatial features (dimensions, furniture arrangement, etc.).

[1191] 4. Generator

[1192] The generator works with an internal database or an external interior design database and uses an AI model (e.g., a Transformer-based generative model) to generate optimal coordination suggestions.

[1193] 5. Terminal

[1194] A smartphone or tablet for users to view the suggestions.

[1195] Processing flow

[1196] After receiving the image and related information from the user, the server first analyzes the spatial features using an image analyzer, primarily using OpenCV to detect feature points and edges in the image, and TensorFlow to recognize objects such as furniture.

[1197] Next, the generator generates an interior coordination proposal based on the space's characteristics and the theme, budget, and color information provided by the user. The generator works in conjunction with an interior design database to make proposals that incorporate the latest designs.

[1198] The generated coordination proposals are sent to the user's device via the server, where the user can check and purchase the proposed furniture and accessories in the physical store.

[1199] Specific examples

[1200] User A takes a picture of the living room using his smartphone, and then enters information such as the theme "Scandinavian style," budget "100,000 yen," and favorite colors "blue and white" through the application and sends it.

[1201] The server uses an image analyzer to analyze the dimensions of the living room and the layout of the existing furniture, and a generator generates optimal furniture and accessory suggestions based on a Nordic theme. For example, the suggestions might look like this:

[1202] Prompt Sentence Examples

[1203] User theme: Nordic style

[1204] Budget: 100,000 yen

[1205] Favorite colors: Blue and white

[1206] The generated suggestions are sent to User A's smartphone, and User A can actually check and touch the suggested items in a physical store. User A can then purchase the products they like on the spot.

[1207] Through this process, users can create their ideal interior coordination both online and in-store.

[1208] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1209] Step 1:

[1210] The user takes a picture of their room using their smartphone. They open a dedicated application on the device, enter relevant information such as the theme (e.g., Scandinavian style), budget (e.g., 100,000 yen), and preferred colors (e.g., blue and white), and send the image and text information to the server. In this step, the input data is the captured image and text information, and the output data is generated to be sent to the server.

[1211] Step 2:

[1212] The server receives the images and related information sent by the user. On the server, an image analysis device receives the images and uses image recognition technology (e.g., OpenCV, TensorFlow) to identify spatial features (dimensions, furniture placement, etc.). In this process, the image stored on the server is used as input data, and analyzed spatial feature information is generated as output.

[1213] Step 3:

[1214] The server sends the analyzed space characteristics information and the theme, budget, and preferred color information received from the user to the generator. The generator receives this information as input data and works with an interior design database to generate optimal interior coordination proposals. Here, coordination proposals are generated using an AI model (e.g., a Transformer-based generative model) based on existing design data in the database. The generated proposal data is generated as output.

[1215] Step 4:

[1216] The server organizes the proposals returned from the generation device and sends them to the user's terminal. In this step, the generated proposal data is used as input data, and data to be sent to the user's terminal is prepared as output.

[1217] Step 5:

[1218] The user checks the proposed interior coordination on the device. The proposals are displayed through a dedicated application, and the user can actually check and touch the proposed items in a physical store. At this stage, the input is the proposal data from the server, and the output is the user's visual and tactile confirmation and purchase of the products.

[1219] Step 6:

[1220] If the user likes an item, they can proceed with the purchase process at a physical store. Support is provided for purchasing the suggested interior items on the spot, and the purchase information is processed through a dedicated application. This process takes the user's purchase intention and payment information as input data, and generates purchase completion data as output.

[1221] The above steps implement a series of operations, allowing users to experience the interior coordination in detail and then check and purchase in a physical store to realize their ideal space.

[1222] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1223] The present invention is a system in which a user acquires an image of a space, sends the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. Furthermore, by combining the present invention with an emotion engine that recognizes the user's emotions, the system can make proposals that take into account the user's emotional state. Specific embodiments are described below.

[1224] User operation

[1225] First, users take pictures of their rooms using a camera such as a smartphone. The images are then uploaded via a dedicated application. This application provides users with an interface for entering relevant information such as theme, color, and budget. The application also incorporates an emotion engine that recognizes the user's emotions, and emotional information is acquired simultaneously when the user uploads the image.

[1226] Server-side processing

[1227] The server receives the image, input information, and emotion information sent by the user. The server then sends the image data to an image analysis device and requests it to analyze the spatial features (e.g., dimensions, existing furniture layout, interior style). The image analysis device extracts the features and returns the analysis results to the server.

[1228] The analysis results and related information are sent to the generator along with the emotion information. The generator then uses this information to generate optimal interior design proposals. The generator then connects with interior design databases around the world to generate item and layout proposals based on the latest trends. Furthermore, the generator takes the user's emotional state into account and makes proposals that correspond to their emotions.

[1229] Sending generated suggestions to users

[1230] The server then organizes the suggestions returned by the generators and sends them to the user's device, where they can be viewed within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[1231] Specific examples

[1232] For example, if User B has a budget of 150,000 yen, a vintage theme, and preferred colors of brown and gold, the user can take a picture of their living room with their smartphone, enter the relevant information into a dedicated app, and send it. At this time, the emotion engine also recognizes User B's real-time emotional state and sends it to the server. The server then uses an image analysis device to analyze the dimensions of User B's living room and the layout of existing furniture, and the generator then suggests optimal furniture and accessories based on the vintage theme.

[1233] The suggestions are sent to User B's smartphone, and User B checks the suggestions within the app. If the suggested items match their emotions and they like them, User B can purchase them within the app. Through this process, users can easily create their ideal interior coordination that is in line with their emotions.

[1234] The above is a specific description of the embodiment of the present invention. This system allows users to receive optimal interior design suggestions that meet their individual needs and emotions without any effort on their part.

[1235] The processing flow will be explained below.

[1236] Step 1:

[1237] The user takes an image of the room using a camera (smartphone camera).

[1238] Step 2:

[1239] The user launches a dedicated application and uploads the captured images to the app.

[1240] Step 3:

[1241] Users input relevant information such as budget, theme, color preferences, etc. through the application interface, and the emotion engine automatically captures emotional information from the user's facial expressions and voice.

[1242] Step 4:

[1243] The user presses the "send" button to send the image, input information, and emotion information to the server.

[1244] Step 5:

[1245] The server receives the image, input information, and emotion information sent by the user.

[1246] Step 6:

[1247] The server sends the received image data to an image analysis device, which analyzes the characteristics of the space (dimensions, arrangement of existing furniture, interior style, etc.).

[1248] Step 7:

[1249] The image analysis device performs the analysis and returns the results to the server.

[1250] Step 8:

[1251] The server transmits the analysis results, information about the theme, budget, and color preferences entered by the user, and emotional information to the generation device.

[1252] Step 9:

[1253] The generator generates optimal furniture and accessory recommendations based on the analysis results and user input. The generator also takes into account the user's emotional information to generate recommendations that are appropriate to their emotions. The generator also connects to a global interior design database, enabling it to provide recommendations based on the latest trends.

[1254] Step 10:

[1255] The generator sends the generated proposal back to the server.

[1256] Step 11:

[1257] The server organizes the proposals and sends them to the user's device.

[1258] Step 12:

[1259] The user checks the proposal on their device (smartphone), which includes a new furniture layout, a list of recommended items, and a link to purchase.

[1260] Step 13:

[1261] The user reviews the suggestions and selects the items they like. The user completes the purchase within the app.

[1262] This allows users to receive basic interior coordination suggestions as well as emotional suggestions. For example, if a user wants to relax, the app will suggest furniture and accessories in calming colors.

[1263] Example 2

[1264] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1265] Conventional interior design suggestion systems have difficulty making suggestions that fully consider the user's individual needs and emotional state. Another problem is the lack of technology that can accurately grasp the characteristics of a space based on image analysis and generate optimal interior design suggestions based on that information. This means that users have to spend a lot of time selecting the interior design themselves, and are unable to obtain satisfactory suggestions.

[1266] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1267] In this invention, the server includes means for capturing images of the space using a photographing device, means for transmitting the images and related information to the server, means for identifying characteristics of the space using an image analyzing device, means for providing a generating device for generating optimal furniture and accessory proposals, means for transmitting the generated proposals to the terminal, and means for acquiring the user's emotional information using an emotion engine and reflecting the information in the generated proposals. This enables the user to easily receive optimal interior proposals that match their individual needs and emotional state based on the captured images, related information, and emotional information.

[1268] A "photography device" is a device that a user uses to capture an image of a space, and typically refers to a smartphone or digital camera equipped with a camera function.

[1269] "Means for transmitting images and related information to a server" refers to a function for transmitting images acquired by a user and information such as theme, budget, and preferred colors to a server via the Internet.

[1270] An "image analysis device" refers to software or hardware that analyzes received image data and extracts spatial characteristics (dimensions, furniture arrangement, interior style, etc.).

[1271] A "generator" is a system that generates optimal furniture and accessory suggestions based on image analysis results and related information, and is typically implemented using a generative AI model.

[1272] The "means for transmitting the generated proposal to the terminal" refers to a communication function for transmitting the interior proposal generated by the generation device to the user's terminal.

[1273] An "emotion engine" is a system for analyzing and acquiring the user's emotional state, and for example, recognizes the user's emotions using facial expression recognition technology.

[1274] The "Interior Design Database" is a database that accumulates data on interior design from around the world and provides information on the latest trends and items.

[1275] MODE FOR CARRYING OUT THE INVENTION

[1276] The present invention is a system in which a user acquires an image of a space, sends the image along with related information to a server, and the server analyzes the image to generate and provide optimal interior coordination proposals. Furthermore, by combining the present invention with an emotion engine that recognizes the user's emotions, it is possible to make proposals that take into account the user's emotional state. Specific embodiments are described below.

[1277] User operation

[1278] First, the user takes a picture of their room using a camera such as a smartphone. The image is then uploaded to a server via a dedicated application. This application provides the user with an interface for entering relevant information such as theme, color, and budget. The application also incorporates an emotion engine, which simultaneously acquires emotional information when the user uploads the image. The emotion engine uses facial expression recognition technology to recognize the user's emotions in real time.

[1279] Server-side processing

[1280] The server receives the image, related information, and emotional information sent by the user. The server then sends the image data to an image analyzer (e.g., an image analysis library such as OpenCV or TensorFlow) to analyze the spatial features (e.g., dimensions, existing furniture arrangement, interior style, etc.). The image analyzer then uses a deep learning model to extract these features from the image.

[1281] The server then sends the analysis results, related information, and emotional information to a generator (such as OpenAI's GPT-4, an example of a generative AI model) to generate optimal interior design suggestions. The generator connects with interior design databases around the world to generate item and layout suggestions based on the latest trends. It also makes suggestions that take the user's emotional state into account.

[1282] Sending generated suggestions to users

[1283] The server then compiles the suggestions returned by the generators and sends them to the user's device, where they can view them within the application. The suggestions include a detailed furniture layout, a list of recommended items, and links to purchase them.

[1284] Specific examples

[1285] For example, if User B sets a budget of 150,000 yen, a vintage theme, and favorite colors of brown and gold, User B takes a picture of their living room with their smartphone, enters the image and related information into a dedicated app, and sends it. At this time, the emotion engine built into the app recognizes User B's emotional state in real time and sends it to the server.

[1286] The server uses OpenCV to analyze the dimensions and existing furniture layout of User B's living room. The analysis results are sent back to the server, which then uses this information to generate optimal furniture and accessory recommendations based on a vintage theme using tools such as OpenAI's GPT-4. Once the recommendations are sent to User B's smartphone, User B can review the recommendations—for example, a vintage-style sofa, lamp, or carpet—in the app, and if they like them, they can simply tap the purchase link to purchase them.

[1287] Prompt Sentence Examples

[1288] "I'd like some suggestions for a vintage-inspired interior. My budget is ¥150,000, my favorite colors are brown and gold, and my current emotional state is relaxed. The image of my living room includes my current furniture arrangement. Please suggest the best arrangement and items."

[1289] The above is a specific embodiment of the present invention. This system allows users to easily create an ideal interior coordination that is in line with their emotions.

[1290] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1291] Step 1:

[1292] Taking photos and uploading images

[1293] input:

[1294] The user takes a picture of their room using the smartphone camera.

[1295] Specific behavior:

[1296] The user opens the camera app on their smartphone and takes a picture that shows the entire room. Then, they open the dedicated application and tap the "Upload" button to import the image into the application.

[1297] output:

[1298] Image data stored on the smartphone is sent to the server via a dedicated application.

[1299] Step 2:

[1300] Enter and submit relevant information

[1301] input:

[1302] Users fill out forms within the application to enter relevant information such as theme, color, budget, etc.

[1303] Specific behavior:

[1304] Users enter information such as the theme (e.g., vintage style), preferred colors (e.g., brown and gold), and budget (e.g., 150,000 yen) into the application's input form and tap the "Submit" button.

[1305] output:

[1306] Relevant information such as theme, color, budget, etc. is sent to the server and associated with the image data.

[1307] Step 3:

[1308] Acquiring emotional information

[1309] input:

[1310] When users upload images, emotional information is collected using facial recognition technology.

[1311] Specific behavior:

[1312] An emotion engine built into the application analyzes the user's face in real time to detect their emotional state (e.g., relaxed).

[1313] output:

[1314] Emotional state data is generated and transmitted to a server.

[1315] Step 4:

[1316] Data reception and analysis

[1317] input:

[1318] The server receives the image data, related information (theme, color, budget), and emotional information sent by the user.

[1319] Specific behavior:

[1320] The server receives the captured image data, related information, and emotion information and temporarily stores them in a database.

[1321] output:

[1322] Stored image data and related information, as well as emotional information.

[1323] Step 5:

[1324] Image analysis

[1325] input:

[1326] Image data stored on the server.

[1327] Specific behavior:

[1328] The server uses image analysis equipment (e.g., OpenCV or TensorFlow) to analyze the image data and extract spatial features (e.g., dimensions, existing furniture arrangement, interior style).

[1329] output:

[1330] Extracted spatial feature data.

[1331] Step 6:

[1332] Proposal generation

[1333] input:

[1334] Analysis results, related information (theme, color, budget) and sentiment information stored on the server.

[1335] Specific behavior:

[1336] The server sends the analysis results, related information, and emotional information to a generator (e.g., OpenAI's GPT-4) to generate optimal interior design proposals. The generator connects to an interior design database to generate item and layout proposals based on the latest interior trends.

[1337] output:

[1338] Generated interior design proposal data (furniture layout, list of recommended items, and purchase links).

[1339] Step 7:

[1340] Submitting generated proposals

[1341] input:

[1342] Generated interior proposal data.

[1343] Specific behavior:

[1344] The server organizes the generated proposals and sends them to the user's terminal.

[1345] output:

[1346] Interior design suggestions displayed on the user's device.

[1347] Step 8:

[1348] Confirmation and decision on proposal contents

[1349] input:

[1350] The offer received by the user's device.

[1351] Specific behavior:

[1352] Users can review the suggestions within the app, select the one they like, and tap the purchase link.

[1353] output:

[1354] Based on the user's purchasing intent, the user is redirected to a page where they can purchase the suggested furniture and accessories.

[1355] In this way, users can easily and quickly receive interior design suggestions that suit their needs and emotions.

[1356] (Application example 2)

[1357] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1358] In modern interior design, users face the challenge of finding suggestions that reflect their emotions and immediate mood. It is also difficult for users to directly and intuitively understand where suggested items will be placed when selecting products in a physical store. Therefore, it is necessary to provide more personalized interior design suggestions by analyzing users' real-time emotions and states and guiding the location of offered items in a physical store.

[1359] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring an image of the space using an imaging device, means for transmitting the image and related information to the server, means for identifying characteristics of the space using an image analysis device in the server, means for including a generation device that generates optimal furniture and accessory suggestions using the analysis results and related information, means for including an emotion engine that recognizes the user's emotional state and adjusts the suggestions based on the emotion information, and means for transmitting the generated suggestions to a terminal and guiding the location of recommended items. This allows the user to receive interior suggestions tailored to their emotional state in real time and be guided in selecting the locations of specific items in a physical store.

[1360] An "imaging device" is a device that allows a user to capture an image of a space, and includes a smartphone, smart glasses, etc.

[1361] "Related information" refers to information related to interior coordination such as budget, theme, and color provided by the user.

[1362] A "server" is a computer system that receives images and information sent by users and performs analysis and generation.

[1363] An "image analysis device" is a device that analyzes transmitted images and identifies the characteristics, dimensions, existing furniture layout, etc. of a space.

[1364] A "generator" is a device that creates optimal furniture and accessory proposals based on analysis results and related information.

[1365] An "emotion engine" is a system that recognizes emotions from a user's facial expressions, voice, etc. and analyzes that information.

[1366] "Terminal" refers to a device that receives and displays interior design proposals generated by a user, and includes smartphones, smart glasses, etc.

[1367] "Means for guiding the location of recommended items" refers to a function that guides the user to the specific location of recommended products or items in the generated suggestions.

[1368] The "interior design database" is a database that stores information on various interior styles and trends, and is a data source that the generation device references when generating proposals.

[1369] In this invention, a user wears smart glasses and walks around a physical store, photographing the products and space inside the store. The smart glasses capture the scenery and products the user sees in real time and save them as images. The user can also input relevant information such as budget, theme, and color preferences into the smart glasses using voice input. At this time, the smart glasses are equipped with an emotion engine that analyzes the user's emotional state from facial expressions and tone of voice to acquire emotional information.

[1370] The server receives the images, related information, and emotion information sent by the user. The server then uses an image analyzer to identify spatial features (dimensions, existing furniture layout, etc.) from the received images. Software libraries such as OpenCV are used for image analysis, and machine learning models such as TensorFlow are used for emotion analysis.

[1371] The server generates optimal interior design suggestions based on the analysis results, related information, and emotional information. The generator is connected to a global interior design database and can suggest furniture and accessories based on the latest trends. The suggestions are adjusted to suit the user's emotional state.

[1372] The generated suggestions are sent to the user's smart glasses in real time. The smart glasses display guides the user to the location of the recommended items, making it easier for the user to identify them in the store. This allows users to easily coordinate interiors that match their emotions and preferences in a physical store.

[1373] Specific examples

[1374] As the user wears the smart glasses and walks around an interior design shop, the glasses take photos of the furniture and decorations in the store. The user inputs "budget 200,000 yen, theme modern, color preference black and white" by voice, and the emotion engine analyzes the user's real-time emotional state and sends it to the server. The server uses an image analysis device to analyze the dimensions and placement of products in the store, and a generator recommends modern-themed furniture and accessories that match the user's emotional information. The glasses' display shows the location of the recommended item along with the message "This item is perfect for your mood!"

[1375] Prompt statement

[1376] "A user visits an interior design shop and uses the smart glasses to input information such as a budget of 200,000 yen, a modern theme, and a color preference of black and white, all while looking around the shop. The smart glasses also analyze the user's emotions and suggest the optimal interior coordination. The suggestions are displayed on the glasses' display in real time, and the user is guided to the location of items based on the suggestions."

[1377] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1378] Step 1:

[1379] A user wears smart glasses and walks around a physical store. The smart glasses' camera captures images of the store in real time. The user inputs relevant information, such as budget, theme, and color preferences, into the smart glasses using voice input. The smart glasses' camera captures image data, and the voice input device captures the relevant information as text. These data are temporarily stored in the smart glasses' memory.

[1380] Input: In-store images, relevant information such as the user's budget, theme, colors, etc.

[1381] Output: Image data, related text data

[1382] Step 2:

[1383] The emotion engine built into the smart glasses analyzes the user's emotional state in real time from their facial expressions and tone of voice. The emotion engine generates emotional information using an AI model that combines facial expression recognition and voice analysis, and stores this information in the smart glasses. The user's emotional information is also generated as text data.

[1384] Input: User's facial expression, tone of voice

[1385] Output: Emotional information text data

[1386] Step 3:

[1387] The smart glasses transmit the acquired image data, related information, and emotional information to a server via wireless communication (Wi-Fi or Bluetooth). The server receives and stores this data.

[1388] Input: Image data, text data of related information, text data of emotional information

[1389] Output: Data stored on the server

[1390] Step 4:

[1391] The server uses an image analysis device to analyze the received image data and identify the characteristics of the space. Software libraries such as OpenCV are used for image analysis. The analysis results, including the dimensions of the space and the layout of existing furniture, are generated as text data and saved on the server.

[1392] Input: Image data

[1393] Output: Text data of spatial features (dimensions, furniture layout, etc.)

[1394] Step 5:

[1395] The server generates optimal interior design proposals based on the analysis results, related information, and emotional information. The generator connects to a global interior design database, which generates furniture and accessory proposals based on the latest trends. The proposals are adjusted to suit the user's emotional state. The final proposals are generated as text and image data.

[1396] Input: Text data of spatial features, text data of related information, text data of emotional information

[1397] Output: Text data and image data of optimal interior design proposals

[1398] Step 6:

[1399] The server sends the generated interior design proposals to the smart glasses via wireless communication (Wi-Fi or Bluetooth). The smart glasses receive the proposals and store them in their internal memory.

[1400] Input: Text data and image data of optimal interior proposals

[1401] Output: Proposal data stored in smart glasses

[1402] Step 7:

[1403] The smart glasses display the suggested items on a screen. The user is guided to locate the suggested items in a physical store. Augmented reality (AR) technology is used to display the guidance. The location of the suggested item is overlaid on the display, and the user can navigate to that location.

[1404] Input: Proposal data stored in smart glasses

[1405] Output: Suggestions and guidance displayed on the smart glasses display

[1406] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1407] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1408] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1409] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1410] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1411] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1412] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1413] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1414] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1415] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1416] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1417] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1418] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1419] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1420] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1421] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1422] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1423] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1424] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1425] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1426] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1427] The following is further disclosed regarding the above embodiment.

[1428] (Claim 1)

[1429] a means for acquiring an image of the space by an imaging device;

[1430] means for transmitting the image and related information to a server;

[1431] a means for identifying spatial characteristics by an image analysis device in the server;

[1432] means for generating optimal furniture and accessory recommendations using the analysis results and related information;

[1433] means for transmitting the generated proposal to a terminal;

[1434] A system including:

[1435] (Claim 2)

[1436] The system of claim 1 , further comprising: means for obtaining information regarding budget, theme, and color.

[1437] (Claim 3)

[1438] 10. The system of claim 1, further comprising means for interfacing with a worldwide database of interior designs to generate suggestions.

[1439] "Example 1"

[1440] (Claim 1)

[1441] a means for acquiring an image of the space by an imaging device;

[1442] means for transmitting the image and related information to a server;

[1443] a means for identifying spatial characteristics by an image analysis device in the server;

[1444] a generating device that uses the analysis results and related information to generate optimal furniture and accessory recommendations utilizing a generative AI model;

[1445] means for transmitting the generated proposal to a terminal;

[1446] A system including:

[1447] (Claim 2)

[1448] The system of claim 1 , further comprising: means for obtaining information regarding budget, theme, and color.

[1449] (Claim 3)

[1450] 10. The system of claim 1, further comprising means for interfacing with a worldwide database of interior designs to generate suggestions.

[1451] "Application Example 1"

[1452] (Claim 1)

[1453] a means for acquiring an image of the space by an imaging device;

[1454] means for transmitting the image and related information to a server;

[1455] a means for identifying spatial characteristics by an image analysis device in the server;

[1456] means for generating optimal furniture and accessory recommendations using the analysis results and related information;

[1457] means for transmitting the generated proposal to a terminal;

[1458] a means for assisting users in browsing and purchasing furniture and interior products in a physical store;

[1459] A system including:

[1460] (Claim 2)

[1461] The system of claim 1 , further comprising: means for obtaining information regarding budget, theme, and color.

[1462] (Claim 3)

[1463] 10. The system of claim 1, further comprising means for interfacing with a worldwide database of interior designs to generate suggestions.

[1464] "Example 2: Combining Emotion Engines"

[1465] (Claim 1)

[1466] a means for acquiring an image of the space by an imaging device;

[1467] means for transmitting the image and related information to a server;

[1468] a means for identifying spatial characteristics by an image analysis device in the server;

[1469] means for generating optimal furniture and accessory recommendations using the analysis results and related information;

[1470] means for transmitting the generated proposal to a terminal;

[1471] a means for acquiring user emotion information using an emotion engine and reflecting the acquired emotion information in generated suggestions;

[1472] A system including:

[1473] (Claim 2)

[1474] The system of claim 1 , further comprising: means for obtaining information regarding budget, theme, and color.

[1475] (Claim 3)

[1476] 10. The system of claim 1, further comprising means for interfacing with a worldwide database of interior designs to generate suggestions.

[1477] "Application example 2 when combining emotion engines"

[1478] (Claim 1)

[1479] a means for acquiring an image of the space by an imaging device;

[1480] means for transmitting the image and related information to a server;

[1481] a means for identifying spatial characteristics by an image analysis device in the server;

[1482] means for generating optimal furniture and accessory recommendations using the analysis results and related information;

[1483] means for adjusting suggestions based on the emotional information, the means comprising an emotion engine that recognizes the emotional state of the user;

[1484] means for transmitting the generated suggestions to a terminal and guiding the location of the recommended items;

[1485] A system including:

[1486] (Claim 2)

[1487] The system of claim 1 , further comprising: means for obtaining information regarding budget, theme, and color.

[1488] (Claim 3)

[1489] 10. The system of claim 1, further comprising means for interfacing with a worldwide database of interior designs to generate suggestions. [Explanation of symbols]

[1490] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for acquiring an image of the space by an imaging device; means for transmitting the image and related information to a server; a means for identifying spatial characteristics by an image analysis device in the server; means for generating optimal furniture and accessory recommendations using the analysis results and related information; means for transmitting the generated proposal to a terminal; A system including:

2. The system of claim 1 , further comprising means for obtaining information regarding budget, theme, and color.

3. The system of claim 1 further comprising means for linking with a worldwide database of interior designs to generate suggestions.

Images