system

Abstract

We provide the system. [Solution] A system including means for taking pictures of a room, means for specifying a genre based on the taken pictures and planning an interior that matches the specified genre, and means for generating image photos of the planned interior.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background Art]

[0002] Patent Document 1 discloses a persona chatbot control method executed by at least one processor, the method comprising the steps of: receiving a user utterance; adding the user utterance to a prompt including an instruction associated with a description of a character of a chatbot; encoding the prompt; and inputting the encoded prompt into a language model to generate a chatbot utterance responding to the user utterance. [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Publication No. 2022-180282 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] Many users who want to change the interior of their room cannot form a concrete image of what kind of interior they should choose. Further, even if a user can form a concrete image, it is difficult to find an interior that matches the image. Furthermore, it is also difficult to coordinate an interior within the user's budget. [Means for Solving the Problem]

[0005] The system takes photos of a room and plans an interior design based on the user's specified genre. It then generates image photos of the planned interior. Furthermore, it directs the user to websites where the corresponding interior items can be purchased based on these images. Finally, it proposes interior design coordinates that fit the user's budget. [Brief explanation of the drawing]

[0006] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 1 of Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2 of Embodiment 2. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2. [Figure 15] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 3 of Example 3. [Figure 16] This is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3. [Figure 17] This is a sequence diagram showing the processing flow of the data processing system in Example 1 of the Form 1 when an emotion engine is combined. [Figure 18] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] This is a sequence diagram showing the processing flow of the data processing system in Example 2 of the Form 2 when an emotion engine is combined. [Figure 20] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] This is a sequence diagram showing the processing flow of the data processing system in Example 3 of the Form 3 when an emotion engine is combined. [Figure 22] This is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. [Figure 23] This is a sequence diagram showing the processing flow of a data processing system in another embodiment. [Modes for carrying out the invention]

[0007] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Further, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of the arithmetic unit include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and a TPU (TENSOR PROCESSING UNIT (Registered Trademark)).

[0010] In the following embodiments, a labeled RAM (Random Access Memory) is a memory that temporarily stores information, and is used as a working memory by a processor.

[0011] In the following embodiments, labeled storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), magnetic tape, and the like.

[0012] In the following embodiments, a labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between a plurality of computers. Examples of communication standards applied to communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (Registered Trademark), Bluetooth (Registered Trademark), and the like.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0014] [First Embodiment]

[0015] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0016] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0019] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0022] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0026] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0027] "Example of form 1"

[0028] One embodiment of the present invention provides a system that includes a camera module for taking pictures of a room, an interior planning module for specifying a genre based on the taken pictures and planning an interior that matches the specified genre, and an image generation module for generating image photos of the planned interior.

[0029] "Example of form 2"

[0030] As another embodiment of the present invention, a system is provided that further includes a site guidance module that guides users to a site where the generated interior image can be purchased. Specifically, it provides a link to an online shopping site where the interior item can be purchased, based on information about the interior item included in the generated image.

[0031] "Example of form 3"

[0032] In yet another embodiment of the present invention, a system is provided that further includes a coordination suggestion module that proposes an interior design that fits the budget provided by the user. Specifically, based on the budget provided by the user, the system selects interior items that can be purchased within that budget and proposes a coordination that combines them.

[0033] The following describes the processing flow for each example form.

[0034] "Example of form 1"

[0035] Step 1: The user takes a picture of the room. This is done using a camera module built into the system.

[0036] Step 2: Based on the photos taken, the user specifies the interior design genre. The genre is specified on the system's user interface.

[0037] Step 3: Plan the interior design to match the specified genre. This planning is done automatically by the system's interior planning module.

[0038] Step 4: Generate image photos of the planned interior. This generation is performed automatically by the system's image generation module.

[0039] "Example of form 2"

[0040] Step 1: Based on the generated interior image photos, the system redirects users to websites where the corresponding interior items can be purchased. This redirection is performed automatically by the system's site redirection module.

[0041] Step 2: Specifically, based on the information about the interior items included in the generated image, provide links to online shopping sites where those items can be purchased. (Example 3)

[0042] Step 1: Based on the budget provided by the user, the system proposes an interior design coordinated to fit that budget. This proposal is automatically generated by the system's coordination proposal module.

[0043] Step 2: Specifically, based on the budget provided by the user, select interior items that can be purchased within that budget and propose a coordinated look using those items.

[0044] (Example 1)

[0045] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0046] In modern living spaces, efficiently proposing interior designs that meet individual preferences and budgets is difficult. Traditional methods often require expert advice and are time-consuming and costly. Furthermore, visualizing interior designs is challenging, making it difficult for clients to form concrete images.

[0047] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0048] In this invention, the server includes means for acquiring an image of a space using a camera; means for analyzing the acquired image and determining the category of the space based on the features in the image; means for planning an interior design that fits the determined category; means for generating a visual representation of the planned interior design using a generative AI model; and means for transmitting the generated visual representation to the user's device. This allows the user to quickly and efficiently visualize interior designs that suit their preferences and budget, and to have a concrete image of them.

[0049] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and smartphones.

[0050] A "spatial image" is digital image data that contains visual information about a specific place or room.

[0051] "Image analysis" is a technique that processes acquired image data to identify features and elements within the image.

[0052] A "spatial category" is a classification that indicates the style or theme of a space, determined based on image analysis.

[0053] "Interior design" is the process of planning the placement of furniture and decorative items to suit a particular space.

[0054] A "generative AI model" is an algorithm that uses artificial intelligence technology to generate new visual representations from data.

[0055] "Visual representation" refers to images or illustrations that visually show the results of the interior design.

[0056] "User's device" refers to a device used to receive and display the generated visual representation, and includes smartphones, computers, and other similar devices.

[0057] An "online platform" is a website or application that provides goods and services via the internet.

[0058] "Budget information" refers to data about the amount of money users can spend on interior design.

[0059] This invention is a system that allows a user to take a photograph of a room and then proposes an interior design based on that photograph. The user uses a terminal's camera to acquire an image of the room. The acquired image is sent from the terminal to a server. The server uses image analysis software to analyze the features in the image and determine the category of the space. OpenCV, a Python image processing library, can be used for this analysis.

[0060] Next, the server plans the interior design based on the determined category. This involves referencing templates in the database and selecting a design that fits the category. The server then uses a generative AI model to generate a visual representation of the planned interior design. Generative AI models such as DALL-E and Stable Diffusion can be used for this generation.

[0061] The generated visual representation is sent from the server to the user's terminal. The user can view the visual representation on their terminal and use it as a reference for interior design. For example, if the user takes a picture of their living room, the server will determine the category to be "modern" and suggest a modern interior design. The generation AI model can generate an appropriate visual representation by inputting a prompt such as, "Generate a modern living room interior design."

[0062] This system allows users to quickly and efficiently visualize interior designs that suit their preferences and budget, giving them a concrete image of what they want.

[0063] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0064] Step 1:

[0065] The user takes an image of the room using the device's camera. The user opens the smartphone's camera app and takes a picture so that the entire room is visible. The captured image is sent from the device to the server. The input is the photo of the room, and the output is the image data sent to the server.

[0066] Step 2:

[0067] The server analyzes the received image data. Using image analysis software, it identifies features within the image and determines the spatial category. OpenCV, a Python image processing library, can be used for this analysis. The input is the image data sent to the server, and the output is the determined spatial category.

[0068] Step 3:

[0069] The server plans the interior design based on the determined category. It refers to templates in the database and selects a design that fits the category. The input is the category of the space, and the output is the planned interior design.

[0070] Step 4:

[0071] The server uses a generative AI model to generate a visual representation of the planned interior design. DALL-E and Stable Diffusion can be used as the generative AI model. The prompt "Generate a modern living room interior design" is entered, and the visual representation is generated. The input is the planned interior design and the prompt, and the output is the generated visual representation.

[0072] Step 5:

[0073] The server sends the generated visual representation to the user's terminal. The user can then view the visual representation on their terminal and use it as a reference for interior design. The input is the generated visual representation, and the output is the visual representation displayed on the user's terminal.

[0074] (Application Example 1)

[0075] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0076] Modern consumers face the challenge of finding the perfect interior design for their living space from a vast array of options. Furthermore, online purchases without in-store inspection can lead to discrepancies between the image and the actual product. Additionally, receiving optimal interior design suggestions that fit within one's budget can be difficult.

[0077] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0078] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations that match the specified category, and means for generating visual information of the planned interior decorations. This allows consumers to visually confirm interiors that suit their own rooms, actually check them in stores, and receive optimal suggestions that fit their budget.

[0079] "Means for acquiring images of a room" refers to a device or method for a user to photograph their living space and import that image data into the system.

[0080] "Means for specifying a category and planning interior decoration to match the specified category" refers to an apparatus or method for analyzing acquired image data, selecting an appropriate interior style based on the user's preferences and the characteristics of the room, and creating a decoration plan that matches that style.

[0081] "Means for generating visual information of planned interior decoration" refers to an apparatus or method for generating an image of the interior in a form that can be visually confirmed by the user, based on the selected interior style.

[0082] "Means for proposing goods" refers to a device or method for proposing specific products or decorative items to a user based on a generated interior image.

[0083] "Means of guiding users to a place where they can actually see the product" refers to a device or method for guiding users to a store or exhibition space so that they can actually see the proposed product or decorative item.

[0084] "Means of providing purchasable information" refers to a device or method that provides users with information to purchase proposed goods or decorative items.

[0085] "A means of proposing interior decoration that matches the presented funds based on the presented funds" refers to a device or method for proposing the optimal interior plan within the budget presented by the user.

[0086] A description of the embodiment for carrying out the invention will be given.

[0087] The system that realizes this invention mainly consists of a user terminal and a server. The user terminal is a mobile information terminal such as a smartphone or tablet, and is equipped with a camera module for acquiring images of the room. The user uses this terminal to take pictures of their room.

[0088] The captured image data is sent from the terminal to the server. The server uses image recognition software (e.g., TENSORFLOW®) to analyze the image data and extract the characteristics of the room. Based on this analysis, the server uses a generative AI model (e.g., OpenAI®'s DALL-E) to select an interior style that suits the user's preferences and the characteristics of the room, and generates visual information of the interior decoration that matches that style.

[0089] The generated visual information is sent to the user's device, where the user can visually confirm it. Furthermore, the server suggests specific products and decorative items to the user based on the generated interior image. These suggestions also include information directing the user to stores or exhibition spaces where they can actually see the products.

[0090] For example, if a user requests a "modern living room," the AI ​​model will receive a prompt such as, "Generate interior design suggestions for a modern living room. The room photos are below." This allows the user to visually see interior designs that suit their room and then actually check them out in a store. The system will also propose an optimal interior design plan based on the user's stated budget.

[0091] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0092] Step 1:

[0093] The user takes a picture of the room using the device's camera module. The input is the image data of the room taken by the user. The output is the captured image data being saved to the device.

[0094] Step 2:

[0095] The device sends the captured image data to the server. The input is the image data stored on the device. The output is the image data transferred to the server.

[0096] Step 3:

[0097] The server analyzes the received image data using image recognition software (e.g., TensorFlow). The input is the image data transferred to the server. Feature extraction is performed as part of the data processing. The output is data indicating the characteristics of the room.

[0098] Step 4:

[0099] The server uses a generative AI model (e.g., OpenAI's DALL-E) to select an interior style that matches the user's preferences and the characteristics of the room, based on room feature data. The input is room feature data. The data calculation involves selecting an interior style. The output is data of the selected interior style.

[0100] Step 5:

[0101] The server generates visual information of the interior decoration based on the selected interior style. The input is data of the selected interior style. As a data processing step, visual information is generated. The output is the generated visual information.

[0102] Step 6:

[0103] The server sends the generated visual information to the user's terminal. The input is the generated visual information. The output is the visual information transferred to the user's terminal.

[0104] Step 7:

[0105] The user reviews the visual information received on the device and visually evaluates the proposed interior. The input is the visual information displayed on the device. The output is the user's evaluation.

[0106] Step 8:

[0107] The server suggests specific products and decorations to the user based on the generated interior image and provides information guiding them to stores or exhibition spaces where they can actually view the products. The input is the generated interior image. The data calculation involves generating product suggestions and guidance information. The output is the suggested product information and guidance information.

[0108] (Example 2)

[0109] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0110] Traditional interior design systems have faced challenges such as difficulty in receiving specific interior design proposals based on the user's desired style and theme, and a lack of information for actually purchasing the suggested interior items. Furthermore, providing optimal interior coordination within the user's budget has also been difficult.

[0111] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0112] In this invention, the server includes means for acquiring images, means for specifying categories based on the acquired images and planning spatial decorations suitable for the specified categories, means for generating visual information of the planned spatial decorations, means for analyzing items included in the generated visual information, and means for providing a connection to an information source from which the analyzed items can be obtained. This allows the user to receive interior design suggestions based on their desired style and to easily purchase the suggested items. It is also possible to provide optimal interior coordination according to the user's budget.

[0113] "Means for acquiring images" refers to a device or method for collecting visual information provided by a user in digital format.

[0114] "Means for specifying categories" refers to an apparatus or method for determining a classification related to a specific style or theme based on acquired images.

[0115] "Means for planning spatial decoration" refers to an apparatus or method for constructing an interior design suitable for a specified category.

[0116] "Means for generating visual information" refers to an apparatus or method for creating a planned spatial decoration as a digital image.

[0117] "Means for analyzing articles" refers to a device or method for identifying interior items contained in generated visual information and extracting their characteristics.

[0118] "Means of providing access to information sources" refers to a device or method for generating and providing to a user a link to an online platform where the analyzed items can be purchased.

[0119] The following systems are conceivable as embodiments for carrying out this invention.

[0120] The user uses a terminal to enter prompts to generate interior design images. These prompts can specify the style or theme the user desires. For example, they might enter prompts such as "modern living room" or "Nordic-style bedroom."

[0121] The terminal sends the prompt message received from the user to the server. The server uses a generative AI model to generate an image of the interior based on the prompt message. This generative AI model can use image generation technologies such as DALL-E or Midjourney.

[0122] The generated images are analyzed by a server. The server uses image recognition technology to identify interior items contained in the images and extract their characteristics. This analysis reveals what the identified items are.

[0123] Next, the server provides a connection to information sources where the identified interior items can be purchased, based on the analysis results. Specifically, it generates and provides links to online shopping platforms. Database search technology can be used to generate these links. For example, it could provide links to platforms such as Amazon or IKEA.

[0124] Users can view images and purchase links generated through their devices. By clicking on the link for an item they are interested in, users can easily access the purchase page and actually buy the item.

[0125] This system allows users to obtain ideas for their desired interior design while simultaneously facilitating the process of actually purchasing those items.

[0126] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0127] Step 1:

[0128] The user uses a terminal to input prompt text for interior design. This prompt text specifically expresses the user's desired style or theme. For example, it might be something like "vintage-style dining room." This prompt text serves as the basic data for subsequent processing.

[0129] Step 2:

[0130] The terminal sends the prompt message received from the user to the server. The server receives this prompt message as input and requests image generation from the generative AI model. The generative AI model generates an image of the interior based on the prompt message. In this process, the content of the prompt message is analyzed and data processing is performed to generate an appropriate image. The generated image is obtained as output.

[0131] Step 3:

[0132] The server analyzes the generated image. It receives an image as input and uses image recognition technology to identify the interior items contained in the photograph. This analysis extracts the type and characteristics of the items. The output provides information about the identified items. Specifically, items such as sofas and tables are identified from the photograph.

[0133] Step 4:

[0134] The server provides a connection to information sources where the identified interior items can be purchased, based on the analysis results. It receives item information as input and uses database search technology to generate links to online shopping platforms. The output is a purchase link. Specifically, links to platforms like Amazon and IKEA are generated.

[0135] Step 5:

[0136] The terminal displays the generated image and purchase link received from the server to the user. The user can view the displayed image and click the link for an item of interest to access the purchase page. It receives the generated image and purchase link as input and provides visual information to the user as output. Specifically, the user can click the link and proceed with the purchase process.

[0137] (Application Example 2)

[0138] Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0139] Modern consumers want to easily find and purchase interior decorations that suit their living spaces. However, in reality, finding the right items often requires a lot of time and effort. Furthermore, the need to compare multiple online platforms to find available items makes efficient purchasing difficult.

[0140] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0141] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations suitable for the specified category, means for generating visual information of the planned interior decorations, and means for identifying items included in the generated visual information and providing connection information to an online platform where the items can be acquired. This makes it possible for users to easily find and efficiently purchase interior decorations that suit their rooms.

[0142] "Means for acquiring images of a room" refers to a function that allows users to take pictures of a room using a mobile device and import that image data into the system.

[0143] "A means of specifying a category and planning interior decoration suitable for that category" refers to a function that selects an interior decoration style or theme based on acquired images and creates a decoration plan that matches it.

[0144] "Means for generating visual information for interior decoration" refers to a function that generates images and graphics to visually represent the planned interior decoration.

[0145] "Means for identifying items and providing connection information to online platforms where those items can be obtained" refers to a function that identifies individual items included in the generated visual information and provides users with links and information to online platforms where those items can be purchased.

[0146] The system for implementing this invention begins with a user acquiring an image of a room using a mobile device and sending that image to a server. The server uses image recognition software (e.g., TensorFlow, OpenCV) to analyze the objects in the image and specify their categories. Based on the specified categories, it uses a generative AI model (e.g., OpenAI's GPT model) to create a suitable interior decoration plan.

[0147] Next, the server generates visual information to visually represent the planned interior decoration. This visual information includes interior items that are suitable for the user's room. The server identifies each item included in the generated visual information and provides connection information to the online platform where they can be obtained. This allows the user to easily purchase items they are interested in.

[0148] As a concrete example, when a user takes a picture of their living room and sends it to the system, the server recognizes items such as a sofa, table, and lamp. For each item, it generates links to online platforms such as Amazon and Rakuten Market and provides them to the user. By clicking on the provided links, the user can directly access the purchase page for the corresponding item.

[0149] An example of a prompt would be, "Please tell me where I can buy the same sofa shown in this picture." By inputting this prompt into the AI ​​model, a link to the appropriate online platform will be generated.

[0150] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0151] Step 1:

[0152] The user takes a picture of the room using a mobile device. The captured image is saved on the device.

[0153] Step 2:

[0154] The terminal sends the captured image to the server. The input is image data of the room, and the output is the transfer of the image data to the server.

[0155] Step 3:

[0156] The server analyzes the received image data using image recognition software (e.g., TensorFlow, OpenCV). The input is image data, and the output is category information of the objects in the image. The server identifies the objects in the image and assigns each category to them.

[0157] Step 4:

[0158] The server uses a generative AI model to create interior decoration plans based on specified categories. The input is category information for items, and the output is an interior decoration plan. The server generates decoration ideas suitable for each category.

[0159] Step 5:

[0160] The server generates visual information to visually represent the planned interior decoration. The input is the interior decoration plan, and the output is visual information (image data). The server generates images to visualize the decoration plan.

[0161] Step 6:

[0162] The server identifies each item included in the generated visual information and provides connection information to the online platform where they can be obtained. The input is visual information, and the output is link information to the online platform. The server generates a purchase link for each item.

[0163] Step 7:

[0164] Users receive link information provided through their device and select items they are interested in. By clicking on the link, users can access the purchase page for the selected item.

[0165] (Example 3)

[0166] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0167] Traditional interior design systems struggled to provide optimal suggestions tailored to the user's budget, and verifying whether the proposed designs were actually affordable was cumbersome. Furthermore, there was a lack of effective means to utilize user feedback to improve the system.

[0168] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0169] In this invention, the server includes means for acquiring images, means for specifying categories based on the acquired images and planning decorative items suitable for the specified categories, means for generating visual information of the planned decorative items, means for receiving budget information from the user, selecting purchasable decorative items based on the budget information and generating suggestions combining the selected decorative items, and means for suggesting the optimal combination of the selected decorative items using a generation AI model. This makes it possible to suggest the optimal interior coordination according to the user's budget, and also makes it easier to confirm that the suggested coordination is actually purchasable. Furthermore, user feedback can be received and used to improve the system.

[0170] "Means for acquiring images" refers to a device or method for visually recording the user's environment or the state of a room.

[0171] "Means of specifying categories" refers to methods for classifying interior styles and themes based on acquired images.

[0172] "Means of planning decorative items" refers to methods for selecting interior items suitable for a specified category and considering their placement.

[0173] "Means of generating visual information" refers to methods for visually representing the planned layout and design of an interior.

[0174] "Means of receiving budget information" refers to methods for obtaining information about the amount of money that users are willing to spend.

[0175] "Methods for selecting decorative items" refers to methods for selecting interior items that are affordable within the user's budget.

[0176] "Means for generating proposals" refers to a method for combining selected decorative items to propose the optimal interior design for the user.

[0177] "Methods using generative AI models" refer to methods that utilize artificial intelligence technology to calculate and propose the optimal combination of selected decorative items.

[0178] "Means of receiving feedback" refers to methods for obtaining opinions and evaluations from users and using them to improve the system.

[0179] A description of embodiments for carrying out this invention will be given.

[0180] The user first accesses the interface using a terminal and enters budget information. The terminal sends this input to the server. The server searches the database based on the budget information received from the user. Specifically, it uses a database management system such as MySQL® to query for interior items that can be purchased within the budget.

[0181] The server generates a coordinated look by combining interior items selected from the search results. This process can utilize a generative AI model. Specifically, the AI ​​model is given a prompt message such as, "Please suggest the best coordinated look using the selected items within the budget," and the AI ​​generates the suggestions.

[0182] The generated outfit suggestions are sent from the server to the terminal. The terminal displays the suggestions to the user. The user can review the suggested outfits and provide feedback as needed. The terminal sends this feedback to the server to help improve the system.

[0183] As a concrete example, if a user specifies a budget of "50,000 yen," the system will select items such as a sofa, table, and lamp that can be purchased within that budget and propose a coordinated look combining them. In this way, the user can receive interior design suggestions that match their budget. The specific processing flow in Example 3 will be explained using Figure 15.

[0184] Step 1:

[0185] The user accesses the interface using their device and enters budget information. Specifically, the user enters the budget amount into a web form or app input field and clicks the "Submit" button. The entered budget information is sent from the device to the server.

[0186] Step 2:

[0187] The server searches the database based on the budget information received from the user. Specifically, it uses a database management system such as MySQL to query for interior items that can be purchased within the budget. The input is the budget information, and the output is a list of items that can be purchased within the budget. The server considers the price information of the items and selects only those items that do not exceed the budget.

[0188] Step 3:

[0189] The server generates a coordinated interior design by combining selected interior items from the search results. A generative AI model can be used for this process. Specifically, the AI ​​model is given the prompt, "Please suggest the best coordinated design using items selected within the budget," and the AI ​​generates the suggestion. The input is a list of items, and the output is the coordinated design suggestion.

[0190] Step 4:

[0191] The server sends the generated outfit suggestions to the terminal. The terminal displays the suggestions to the user. Specifically, the terminal displays images and descriptions of the outfits on a web page or app screen, allowing the user to review them. The input is the outfit suggestions, and the output is the visual display to the user.

[0192] Step 5:

[0193] Users can review suggested outfits and provide feedback. The device sends this feedback to the server to help improve the system. Specifically, users can choose options such as "I like this outfit" or "I want to see other suggestions." The input is user feedback, and the output is information related to system improvements.

[0194] (Application Example 3)

[0195] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0196] Modern consumers face the challenge of achieving optimal interior design within a limited budget. Furthermore, selecting and combining interior items requires specialized knowledge, making it a high hurdle for the average consumer. Therefore, there is a need for a system that can easily propose optimal interior design solutions within a given budget.

[0197] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0198] In this invention, the server includes means for acquiring images of a space using a camera; means for selecting a category based on the acquired images and planning decorative items that fit the selected category; and means for receiving budget information from the user, selecting decorative items that can be purchased within that budget, and proposing the optimal combination. This makes it possible for consumers to easily achieve the optimal interior design within their budget.

[0199] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and smartphones.

[0200] "Spatial images" are photographs or image data that contain visual information about a room or a specific area.

[0201] A "category" is a standard used to classify interior items, and is selected based on style and purpose.

[0202] "Decorative items" refer to furniture, accessories, and other items used in interior design.

[0203] "Planning" is the process of creating an interior design by combining decorative items that fit the selected category.

[0204] A "visualized image" is an image that visually represents the planned arrangement and combination of decorative items.

[0205] "Budget information" refers to data about the amount of money users can spend on interior design.

[0206] An "online platform" refers to a website or application that allows users to purchase goods over the internet.

[0207] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and select decorative items that can be purchased within a budget.

[0208] The system for implementing this invention mainly consists of a server and a user terminal. The server has the function of acquiring images of a space using a camera and receives image data transmitted from the user terminal. Based on the acquired images, the server selects a category and processes items to plan decorative items that fit the selected category. In this process, the server uses a generative AI model to consider budget information provided by the user and selects decorative items that can be purchased within the budget.

[0209] The server generates a visualization of the planned decorations and sends it to the user's terminal. The user can view the visualization on their terminal and purchase the suggested decorations through the online platform. The server generates and provides a link to the online platform to the user.

[0210] As a concrete example, if a user takes a picture of their living room with their smartphone and enters a budget of 50,000 yen, the server analyzes the image and selects a category suitable for the living room. The generating AI model then selects decorative items such as sofas, tables, and curtains that can be purchased within 50,000 yen and suggests the optimal combination. The user can then review the suggested coordination and purchase the items they like.

[0211] An example of a prompt for a generating AI model is, "Please select interior items that can be purchased for a budget of 50,000 yen and suggest a living room coordination." This prompt allows the AI ​​model to provide optimal suggestions tailored to the user's needs.

[0212] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[0213] Step 1:

[0214] The user takes a picture of the room using their smartphone and sends it to the server through the application. The input is the image data of the room, and the output is the transfer of the image data to the server. In this step, the smartphone's camera function is used to acquire the image and upload it to the server via the internet.

[0215] Step 2:

[0216] The server analyzes the received image data and uses image recognition technology to select a room category. The input is image data sent by the user, and the output is the selected category information. The server uses an image analysis algorithm to extract room features and determine the appropriate category.

[0217] Step 3:

[0218] The user enters budget information through the application and sends it to the server. The input is the budget information specified by the user, and the output is the transfer of budget data to the server. In this step, the user specifies the budget using the application's input form and sends it to the server.

[0219] Step 4:

[0220] The server uses a generative AI model to select purchasable decorative items based on the chosen category and budget information. The input is category information and budget information, and the output is a list of selected decorative items. The server inputs prompts into the AI ​​model to select the most suitable decorative item within the budget.

[0221] Step 5:

[0222] The server generates a visualization image based on the selected ornaments and sends it to the user terminal. The input is a list of ornaments, and the output is a visualization image. The server uses a graphics generation algorithm to visualize the arrangement of the ornaments and generate the image.

[0223] Step 6:

[0224] The user views visualized images on their device and selects their preferred decorative items. The input is the visualized images, and the output is the user's selection information. The user uses the application interface to review the suggested decorative items and select the items they wish to purchase.

[0225] Step 7:

[0226] The server generates and provides to the user a link to the online platform corresponding to the selected decorative item. The input is the user's selection information, and the output is a link to the online platform. The server generates a link to the purchase page for the selected item and sends it to the user.

[0227] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0228] "Example of form 1"

[0229] One embodiment of the present invention is a system that combines a user emotion engine with a system that recognizes the user's emotions. This system takes a photo of a room and specifies a genre based on the photo. It plans an interior that matches the specified genre and generates an image of the planned interior. Furthermore, based on the generated image of the interior, it guides the user to a website where the corresponding interior can be purchased. Finally, based on the budget provided by the user, it proposes an interior coordination that fits the budget. In addition to this series of steps, a user emotion engine is incorporated to recognize the user's emotions. Specifically, it estimates the user's emotions from their facial expressions, tone of voice, and speed of operation when they operate the system, and proposes an interior that matches that emotion. For example, if the user is showing feelings of joy, it will propose an interior with a bright and cheerful atmosphere. Conversely, if the user is feeling down, it will propose a calm and soothing interior. This makes it possible to provide more personalized interior proposals that respond to the user's emotions.

[0230] "Example of form 2"

[0231] One embodiment of the present invention is a system that combines a user emotion engine with a system that recognizes the user's emotions. This system takes a photo of a room and specifies a genre based on the photo. It plans an interior that matches the specified genre and generates an image of the planned interior. Furthermore, based on the generated image of the interior, it guides the user to a website where the corresponding interior can be purchased. Finally, based on the budget provided by the user, it proposes an interior coordination that fits the budget. In addition to this series of steps, a user emotion engine is incorporated to recognize the user's emotions. Specifically, it estimates the user's emotions from their facial expressions, tone of voice, and speed of operation when they operate the system, and proposes an interior that matches that emotion. For example, if the user is showing feelings of joy, it will propose an interior with a bright and cheerful atmosphere. Conversely, if the user is feeling down, it will propose a calm and soothing interior. This makes it possible to provide more personalized interior proposals that respond to the user's emotions.

[0232] "Example of form 3"

[0233] One embodiment of the present invention is a system that combines a user emotion engine with a system that recognizes the user's emotions. This system takes a photo of a room and specifies a genre based on the photo. It plans an interior that matches the specified genre and generates an image of the planned interior. Furthermore, based on the generated image of the interior, it guides the user to a website where the corresponding interior can be purchased. Finally, based on the budget provided by the user, it proposes an interior coordination that fits the budget. In addition to this series of steps, a user emotion engine is incorporated to recognize the user's emotions. Specifically, it estimates the user's emotions from their facial expressions, tone of voice, and speed of operation when they operate the system, and proposes an interior that matches that emotion. For example, if the user is showing feelings of joy, it will propose an interior with a bright and cheerful atmosphere. Conversely, if the user is feeling down, it will propose a calm and soothing interior. This makes it possible to provide more personalized interior proposals that respond to the user's emotions.

[0234] The following describes the processing flow for each example form.

[0235] "Example of form 1"

[0236] Step 1: The user takes a photo of the room.

[0237] Step 2: The system specifies a genre based on the photos it has taken.

[0238] Step 3: The system plans an interior design that matches the specified genre and generates image photos of the planned interior.

[0239] Step 4: Based on the interior image photos generated by the system, users are directed to a website where the corresponding interior items can be purchased.

[0240] Step 5: Based on the budget provided by the user, the system proposes interior design ideas that fit within that budget.

[0241] Step 6: The system uses an emotion engine to recognize the user's emotions, estimating their feelings from their facial expressions, tone of voice, and speed of operation while they interact with the system.

[0242] Step 7: The system suggests interior design elements that match the user's estimated emotions. For example, if the user is expressing joy, the system suggests a bright and cheerful interior design. Conversely, if the user is feeling down, the system suggests a calming and soothing interior design.

[0243] (Example 1)

[0244] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0245] In modern living spaces, there is a demand for personalized interior design proposals that cater to the individual user's emotions and budget. However, conventional systems struggle to provide interior design proposals that take user emotions into account, and they also have difficulty quickly providing optimal proposals that fit within the budget.

[0246] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0247] In this invention, the server includes means for acquiring images of a space using a camera; means for identifying categories based on the acquired images and planning interiors that fit the identified categories; means for generating visual information of the planned interiors; and means for recognizing the user's emotions using an emotion analysis device and adjusting interior design suggestions based on the recognized emotions. This enables personalized interior design suggestions that are tailored to the user's emotions and budget.

[0248] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and sensors.

[0249] A "spatial image" is digital image data that includes visual information of a room or a specific location.

[0250] "Category" refers to the genre or style of interior design classified based on the characteristics of the space.

[0251] "Interior design" refers to all aspects of design, including the decoration of rooms and spaces, and the arrangement of furniture.

[0252] "Planning methods" refer to the processes and methods for designing and proposing interiors that fit a specific category.

[0253] "Visual information" refers to images and illustrations that visually represent the planned interior design.

[0254] An "emotion analysis device" is a device designed to recognize the user's emotions and has the function of analyzing facial expressions, tone of voice, operation speed, etc.

[0255] "User emotions" refers to the psychological state or feelings that users exhibit when operating the system.

[0256] "Means of adjusting proposals" refers to methods or processes for modifying or optimizing interior design proposals based on perceived emotions.

[0257] The following system is constructed as an embodiment of this invention.

[0258] The server first receives an image of the space transmitted from the terminal. This image is taken by the user using a camera. The server uses a generative AI model to analyze the received image and extract spatial features from it. Based on these features, the server identifies a category and plans an interior design that fits that category.

[0259] The interior planning module is used for interior design planning. This module searches the database for relevant interior items and generates a list of furniture and decorations that fit the categories. The server then uses the image generation module to generate visual information of the planned interior. This visual information is intended to provide the user with a concrete image of the interior.

[0260] Furthermore, the server uses an emotion analysis device to recognize the user's emotions. It analyzes data such as the user's facial expressions, tone of voice, and operation speed to estimate the user's emotions. Based on this emotional information, the server adjusts its interior design suggestions. For example, if the user is showing an emotion of joy, the server sends a prompt message to the AI ​​model saying, "Please suggest an interior with a bright and cheerful atmosphere," and then makes a suggestion.

[0261] As a concrete example, here is an example of a prompt message to be input into a generative AI model: "Analyze the room photos taken by the user and specify a genre. Plan an interior design that matches that genre and generate an image photo. Also, recognize the user's emotions and provide interior design suggestions that match those emotions."

[0262] In this way, the server can provide personalized interior design suggestions tailored to the user's emotions and budget.

[0263] The flow of the specific processing in Example 1 will be explained using Figure 17.

[0264] Step 1:

[0265] The user takes a picture of the room using the camera on their device. The captured image is sent from the device to the server. The input is the image data of the room, and the output is the transmission of the image data to the server.

[0266] Step 2:

[0267] The server analyzes the received image data. Using a generative AI model, it extracts spatial features from the image and identifies categories. The input is the received image data, and the output is the identified category information.

[0268] Step 3:

[0269] The server uses an interior planning module to plan interiors that fit a specified category. It searches the database for relevant interior items and generates a list of furniture and decorations that match the category. The input is category information, and the output is a list of interior items.

[0270] Step 4:

[0271] The server uses an image generation module to generate visual information of the planned interior. It sends a prompt message to the generation AI model saying, "Create an image based on this interior list," and creates a realistic image. The input is a list of interior items, and the output is visual information of the interior.

[0272] Step 5:

[0273] The server recognizes the user's emotions using an emotion analysis device. It analyzes data such as the user's facial expressions, voice tone, and operation speed to estimate their emotions. The input is the user's operation data, and the output is the estimated emotion information.

[0274] Step 6:

[0275] The server adjusts interior design suggestions based on the recognized emotional information. For example, if the user indicates a feeling of joy, the server sends a prompt message to the AI ​​model saying, "Please suggest an interior with a bright and cheerful atmosphere," and then makes a suggestion. The input is emotional information, and the output is an adjusted interior design suggestion.

[0276] Step 7:

[0277] Based on the generated visual information of the interior, the server provides guidance to information sources where the corresponding interior can be purchased. A user can check this information through a terminal and purchase a product of interest. The input is visual information of the interior, and the output is guidance on purchase information.

[0278] (Application Example 1)

[0279] Next, Application Example 1 of Embodiment 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart device 14 is referred to as a "terminal".

[0280] Modern consumers demand personalized interior decoration proposals that match their individual emotions and budgets. However, conventional interior decoration proposal systems have had problems in that it is difficult to provide proposals that consider the user's emotions, and optimal proposals according to the budget cannot be provided.

[0281] The specifying process performed by the specifying processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0282] In the present invention, the server includes: means for acquiring an image of a room; means for specifying a category based on the acquired image and planning an interior decoration matching the specified category; means for generating visual information of the planned interior decoration; and means for recognizing a user's emotion and proposing an interior decoration according to the emotion. This enables provision of personalized interior decoration proposals that match the user's emotion and budget.

[0283] The "means for acquiring an image of a room" is an apparatus or method for photographing the state of a room and acquiring image data thereof.

[0284] The "means for specifying a category" is an apparatus or method for classifying the style or theme of interior decoration based on the acquired image data.

[0285] "Means for planning interior decoration" refers to an apparatus or method for devising the appropriate arrangement of furniture and decorative items based on a specified category.

[0286] "Means for generating visual information" refers to a device or method for visually representing the image of a planned interior decoration.

[0287] "Means for recognizing user emotions" refers to a device or method for estimating a user's emotions from their facial expressions, tone of voice, etc.

[0288] "Means for proposing interior decoration" refers to a device or method for presenting optimal interior decoration in accordance with the recognized emotions of the user.

[0289] The system for implementing this invention begins with the user acquiring images of a room using a smartphone or smart glasses. The terminal uses a camera module to capture images of the room and sends the data to a server. The server analyzes the acquired images using image recognition technology and specifies the category of interior decoration. In this process, image recognition software such as Google® Cloud Vision API can be used.

[0290] Next, the server plans the interior decoration based on the specified category. Visual information of the planned interior decoration is generated using a generative AI model. By utilizing generative AI models such as OpenAI GPT-3 (registered trademark), it is possible to provide users with visually appealing images of interior decoration.

[0291] Furthermore, the device analyzes the user's facial expressions and tone of voice to recognize their emotions. Using technologies such as Microsoft® Azure® Face API and IBM Watson® Speech to Text, it can estimate the user's emotions. The server then suggests optimal interior decorations based on the recognized emotions.

[0292] For example, if a user takes a picture of their living room using smart glasses, the server analyzes the image and assigns it the "modern style" category. If the server detects that the user is smiling, it suggests furniture and decorations in bright colors. An example of a prompt might be, "Suggest products that match the interior of this photo. The user's emotion is joy."

[0293] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[0294] Step 1:

[0295] The user takes pictures of the room using a smartphone or smart glasses. The input is image data acquired through the camera module, and the output is that same image data. The image is captured when the user presses the capture button.

[0296] Step 2:

[0297] The device sends the captured image data to the server. The input is the image data acquired in step 1, and the output is the image data sent to the server. The device uploads the data over the network.

[0298] Step 3:

[0299] The server analyzes the received image data using image recognition technology and specifies the category of interior decoration. The input is the image data sent to the server, and the output is the specified category. The server uses the Google Cloud Vision API to analyze the image and determine the category.

[0300] Step 4:

[0301] The server plans interior decoration based on a specified category and generates visual information using a generative AI model. The input is the specified category, and the output is the generated visual information. The server uses OpenAI GPT-3 to generate visually attractive images of interior decoration.

[0302] Step 5:

[0303] The terminal analyzes the user's facial expression and voice tone to recognize emotions. The input is the user's facial expression data and audio data, and the output is the recognized emotion. The terminal uses Microsoft Azure Face API and IBM Watson Speech to Text to estimate emotions.

[0304] Step 6:

[0305] The server proposes an optimal interior decoration according to the recognized emotion. The input is the recognized emotion and the generated visual information, and the output is the proposed interior decoration. The server adjusts the content of the proposal based on the user's emotion.

[0306] (Example 2)

[0307] Next, Example 2 of Embodiment 2 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart device 14 is referred to as a "terminal".

[0308] Modern consumers demand personalized interior decoration proposals according to individual emotions and budgets, but there is a problem that it is difficult to efficiently achieve this with conventional systems. There is also a demand for facilitating access to purchasable decorative items.

[0309] The specifying processing performed by the specifying processing unit 290 of the data processing device 12 in Example 2 is realized by the following respective means.

[0310] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images, means for planning interior decoration that matches the specified category, means for generating visual information of the planned interior decoration, means for analyzing information on decorative items included in the generated visual information, means for providing links to information sources where decorative items can be purchased based on the analysis results, and means for recognizing the user's emotions and suggesting interior decoration that corresponds to those emotions. This makes it possible to suggest personalized interior decoration that corresponds to the user's emotions and budget, and also makes it easier to access purchasable decorative items.

[0311] "Means for acquiring images of a room" refers to a device or method for receiving a photograph of a room taken by a user as digital data.

[0312] "Means for specifying a category" refers to a device or method for analyzing acquired images of a room and determining an appropriate interior genre or style based on those images.

[0313] "Means for planning interior decoration" refers to an apparatus or method for constructing an appropriate interior design based on a specified category.

[0314] "Means for generating visual information" refers to an apparatus or method for creating images or graphics to visually represent a planned interior decoration.

[0315] "Means for analyzing information on decorative items" refers to a device or method for identifying individual interior items included in the generated visual information and obtaining detailed information therefrom.

[0316] "Means of providing links to information sources" refers to devices or methods for providing users with links to online platforms or stores where the analyzed ornaments can be purchased.

[0317] "Means for recognizing user emotions" refers to a device or method for analyzing a user's facial expressions, tone of voice, operation speed, etc., to estimate the user's emotional state.

[0318] "Means of proposing interior decoration" refers to a device or method for presenting an optimal interior design to a user based on their perceived emotions and budget.

[0319] The following systems are conceivable as embodiments for carrying out this invention.

[0320] The server receives images of a room taken by the user using their device. The images are sent to the server as digital data, and the server analyzes the images using image recognition technology. This analysis determines the appropriate interior category based on the room's style and characteristics. For example, if the image contains a lot of wooden furniture, the "natural" category will be assigned.

[0321] Next, the server uses a generative AI model to plan the interior based on the specified category. The generative AI model is given specific instructions as prompts, such as "Please suggest an interior for a natural living room." Based on these prompts, the AI ​​model generates an interior image as visual information.

[0322] The generated interior image is sent from the server to the terminal and displayed to the user. The terminal analyzes each interior item included in the image and provides links to sources where each item can be purchased. For example, if the sofa shown in the image is available for purchase at a specific online store, a link to that store is presented to the user.

[0323] Furthermore, the device uses an emotion engine to analyze the user's facial expressions, tone of voice, and operation speed to estimate the user's emotions. Based on this emotion information, the server provides interior design suggestions that match the user's emotions. For example, if the user is expressing joy, the server will suggest an interior design with a bright and cheerful atmosphere.

[0324] In this way, users can easily find personalized interiors that suit their mood and budget.

[0325] The flow of the specific processing in Example 2 will be explained using Figure 19.

[0326] Step 1:

[0327] The user takes a picture of the room using their smartphone or camera. The captured image is saved as digital data on the device. The device then sends this image data to the server. The input is the image data of the room, and the output is the transmission of the image data to the server.

[0328] Step 2:

[0329] The server analyzes the received image data of the room. Using image recognition technology, it extracts features from the image and assigns an interior category. For example, if the image contains a lot of wooden furniture, the "natural" category will be assigned. The input is image data of the room, and the output is the assigned interior category.

[0330] Step 3:

[0331] The server uses a generative AI model to plan interiors based on a specified category. A prompt, such as "Please suggest a natural living room interior," is input to the generative AI model. The AI ​​model then generates interior images based on this prompt. The input consists of an interior category and a prompt, while the output is an interior image.

[0332] Step 4:

[0333] The server sends the generated interior image to the terminal. The terminal displays the received image to the user. The input is the interior image, and the output is the display of the image to the user.

[0334] Step 5:

[0335] The device analyzes each interior item included in the displayed interior image. Based on the analysis results, it generates and provides to the user links to sources where each item can be purchased. For example, if the sofa shown in the image is available for purchase at a specific online store, a link to that store will be displayed. The input is an interior image, and the output is a purchase link.

[0336] Step 6:

[0337] The terminal analyzes the user's facial expressions, tone of voice, and operation speed using an emotion engine to estimate the user's emotions. Based on this emotion information, the server provides interior design suggestions that correspond to the user's emotions. For example, if the user is expressing joy, it will suggest an interior design with a bright and cheerful atmosphere. The input is the user's emotion data, and the output is an interior design suggestion that corresponds to those emotions.

[0338] (Application Example 2)

[0339] Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0340] Conventional interior design proposal systems have challenges in providing personalized suggestions that cater to users' emotions and budgets, and in offering insufficient means for users to easily purchase the suggested decorative items. Furthermore, because interior design suggestions are not based on users' emotions, it is difficult to improve user satisfaction.

[0341] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0342] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations suitable for the specified category, means for generating visual information of the planned interior decorations, and means for recognizing the user's emotions and suggesting interior decorations corresponding to those emotions. This makes it possible to suggest personalized interior decorations that match the user's emotions and budget, and further makes it possible to easily purchase the suggested decorative items.

[0343] "Method for acquiring images of the room" refers to a function that allows users to take photos of the room using their smartphones or cameras and import that image data into the system.

[0344] "Means for specifying a category and planning interior decoration suitable for the specified category" refers to a function that analyzes acquired images of a room, identifies the interior style and theme, and then selects and plans appropriate interior decoration based on that.

[0345] "Means for generating visual information of interior decoration" refers to a function that generates images and graphics to visually represent the planned interior decoration.

[0346] "A means of recognizing the user's emotions and proposing interior decorations that correspond to those emotions" refers to a function that analyzes the user's facial expressions and tone of voice to estimate their emotions and then proposes interior decorations that are appropriate for those emotions.

[0347] "Means of directing users to online platforms" refers to a function that provides links to online shopping sites where related decorative items can be purchased, based on the generated visual information of the interior decorations.

[0348] "A means of proposing interior decorations that fit the budget provided" refers to a function that selects and proposes interior decorations that are available for purchase within the budget range provided by the user.

[0349] The system for implementing this invention consists of a user terminal and a server. The user terminal is a portable information terminal such as a smartphone or tablet, equipped with a camera and microphone. The server provides computing resources for image processing, emotion recognition, and interior design proposals using generative AI models.

[0350] The user's device uses a camera to acquire images of the room. The acquired images are sent to a server, which analyzes the images using image recognition software (e.g., TensorFlow, OpenCV) and assigns a category. Based on the assigned category, the server uses a generative AI model (e.g., DALL-E, Stable Diffusion) to generate visual information for suitable interior decorations.

[0351] Furthermore, the user's device uses a microphone and camera to capture the user's facial expressions and voice tone, and sends this information to the server. The server uses emotion recognition software (e.g., Affectiva, Microsoft Azure Emotion API) to estimate the user's emotions and suggests interior decorations that correspond to those emotions.

[0352] The proposed interior decorations are displayed as visual information on the user's device, and links to online platforms where the related decorations can be purchased are provided. Based on the presented budget, the user can select interior decorations that are available within that budget.

[0353] For example, if a user takes a photo of their room with their smartphone and the app recognizes that the user wants to relax, the server will suggest calming color schemes for the interior and display a link to purchase the items. An example of a prompt would be, "Please suggest relaxing interior items that would suit this room."

[0354] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[0355] Step 1:

[0356] The user uses the device's camera to capture images of the room. The captured images are sent from the device to the server. The input is the image data of the room, and the output is the transmission of the image data to the server.

[0357] Step 2:

[0358] The server analyzes the received image data using image recognition software (e.g., TensorFlow, OpenCV). The analysis results in a category based on the room's style and theme. The input is image data of the room, and the output is the specified category information.

[0359] Step 3:

[0360] The server generates visual information for suitable interior decorations using a generative AI model (e.g., DALL-E, Stable Diffusion) based on the specified category information. The input is category information, and the output is visual information for interior decorations.

[0361] Step 4:

[0362] The user uses the device's microphone and camera to capture their facial expressions and voice tone, and sends this data to the server. The input is the user's facial expression and voice tone data, and the output is the data transmission to the server.

[0363] Step 5:

[0364] The server analyzes the received facial expression and voice tone data using emotion recognition software (e.g., Affectiva, Microsoft Azure Emotion API) to estimate the user's emotion. The input is facial expression and voice tone data, and the output is the estimated emotion information.

[0365] Step 6:

[0366] The server suggests interior design elements suitable for the user based on estimated emotional information. The suggested interior design elements are displayed as visual information on the terminal. The input consists of emotional information and visual information of the interior design elements, and the output is the suggested display to the user.

[0367] Step 7:

[0368] Users can view visual information about interior decorations displayed on their device and proceed with a purchase by clicking on links to online platforms where related decorations can be purchased. The input is the user's selection, and the output is access to the online platform.

[0369] (Example 3)

[0370] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0371] Modern consumers demand personalized interior design tailored to their individual emotions and budgets, but traditional systems struggle to efficiently achieve this. In particular, adjusting emotionally-based suggestions and selecting the most suitable decorative items within a given budget are difficult.

[0372] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0373] In this invention, the server includes means for acquiring images of a space using a camera, means for determining the category of the space based on the acquired images, means for planning decorative items that fit the determined category, means for analyzing the user's emotions, and means for adjusting the decorative item suggestions based on the analyzed emotions. This makes it possible to propose personalized spatial decorations that suit the user's emotions and budget.

[0374] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and smartphones.

[0375] A "space category" is a classification that indicates the style or theme of a space, determined based on the acquired images, and refers to genres such as "modern" or "classic."

[0376] "Decorative items" are items used to decorate a space, and include furniture, interior accessories, and works of art.

[0377] "Visual representation" refers to images or illustrations that visually show the planned arrangement and combination of decorative items.

[0378] "User emotions" refer to the psychological state a user exhibits when using the system, and are estimated from factors such as facial expressions, tone of voice, and speed of operation.

[0379] "Analyzing" is the process of analyzing data to extract specific information, and is done to understand emotions or the content of images.

[0380] "Adjusting the proposal" means changing the selection and placement of decorative items to suit the user's needs based on the analyzed information.

[0381] "Information provision" means providing links to online stores or retailers where the decorative items can be obtained, based on the generated visual representation.

[0382] "Financial information" refers to information about the budget provided by the user, and serves as a constraint in the selection of decorative items.

[0383] This system allows users to take pictures of their spaces and receive suggestions for decorative items that suit their budget and preferences. Users take photos of their rooms using a terminal and upload the images to the system. The server uses image analysis software to determine the category of the space. This analysis uses image recognition technology to identify categories such as "modern" or "classic" based on the room's color scheme and furniture style, for example.

[0384] Next, the server selects appropriate decorative items from the database based on the determined category and the user's budget. The server filters the price information of the decorative items to find the best combination within the budget. In this process, a generative AI model is used to generate visual representations of the decorative items. The generative AI model receives a prompt and generates an image that meets the specified conditions.

[0385] Furthermore, the server analyzes the user's facial expressions, tone of voice, and operation speed obtained from the terminal, and uses an emotion engine to estimate the user's emotions. Based on the analyzed emotions, the server adjusts its suggestions for accessories. For example, if the user wants to relax, it will suggest accessories in calming colors.

[0386] Finally, the server presents the user with a visual representation of the generated ornament and provides a link to the source of information where the ornament can be obtained. The user can click the link through their device and proceed with the purchase.

[0387] As a concrete example, if a user inputs "My budget is 50,000 yen, and I would like a relaxing living room," the server will suggest relaxing decorative items that can be purchased for under 50,000 yen and generate an image of them using a generation AI model. An example of a prompt would be, "Please suggest relaxing living room decorative items that can be purchased for under 50,000 yen." The specific processing flow in Example 3 will be explained using Figure 21.

[0388] Step 1:

[0389] The user takes a picture of the room using their device and uploads the image to the system. The input is the image of the room taken by the user. The output is the image data sent to the server. Specifically, the user uses their smartphone's camera function to take a picture of the entire room and uploads the image through the application.

[0390] Step 2:

[0391] The server inputs the received image data into image analysis software to determine the category of the space. The input is image data sent by the user. The output is the category information of the analyzed space. Specifically, the server uses an image recognition algorithm to analyze the color tones and furniture styles within the image and identify categories such as "modern" or "classic."

[0392] Step 3:

[0393] The server takes the determined category and user budget information as input and selects appropriate decorative items from the database. The input is the spatial category information and the user's budget information. The output is a list of decorative items that can be purchased within the budget. Specifically, the server executes a database query and filters out decorative items that fit the category and are within the budget.

[0394] Step 4:

[0395] The server uses a generative AI model to generate visual representations of selected ornaments. The input is a list of selected ornaments. The output is the generated visual representation of the ornaments. Specifically, the server inputs prompt statements into the generative AI model and generates images that meet the conditions.

[0396] Step 5:

[0397] The server uses an emotion engine to analyze the user's emotions, taking as input the user's facial expressions, voice tone, and operation speed obtained from the terminal. The input consists of the user's facial expression data and voice data. The output is the analyzed user emotion information. Specifically, the server uses a machine learning model to analyze the characteristics of facial expressions and voice and estimate the user's emotions.

[0398] Step 6:

[0399] The server adjusts accessory suggestions based on the analyzed emotional information. The input is the user's emotional information and a visual representation of the accessory. The output is the adjusted accessory suggestions. Specifically, the server changes the color and style of the accessories according to the user's emotions, providing the most suitable suggestions.

[0400] Step 7:

[0401] The server presents the user with a visual representation of the generated ornament and provides links to information sources where the ornament can be obtained. The input is a suggested, customized ornament. The output is the visual representation of the ornament and link information presented to the user. Specifically, the server sends the image and link to the terminal, through which the user can proceed with the purchase process.

[0402] (Application Example 3)

[0403] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0404] Modern consumers are seeking personalized interior design suggestions that cater to their individual emotions and budgets. However, traditional systems struggle to provide suggestions that consider user emotions and offer optimal solutions within a given budget. Furthermore, there is a lack of easy ways for consumers to purchase the suggested interior design items.

[0405] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0406] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations that match the specified category, means for generating visual information of the planned interior decorations, means for recognizing the user's emotions, means for suggesting interior decorations that correspond to the emotions based on the recognized emotions, means for suggesting interior decorations that match the budget provided by the user, and means for generating suggestions using a generative AI model. This makes it possible to suggest personalized interior decorations that correspond to the user's emotions and budget, and furthermore, makes it possible to easily purchase the suggested decorations.

[0407] "Method for acquiring images of the room" refers to a function that allows users to take photos of the room using their smartphones or cameras and import that image data into the system.

[0408] The "means for specifying categories" refer to a function that classifies the style and theme of interior decoration based on acquired images of rooms and selects the appropriate category.

[0409] "Means of planning interior decoration" refers to the function of selecting furniture and decorative items suitable for a room based on a specified category, and constructing the overall design.

[0410] "Means for generating visual information" refers to functions for visually representing the planned interior design as images or 3D models.

[0411] "Means of recognizing user emotions" refers to functions that analyze the user's facial expressions and tone of voice to estimate their emotional state.

[0412] "A means of proposing interior decoration that responds to emotions" refers to a function that selects and proposes interior decoration with an appropriate atmosphere and style based on the recognized emotions of the user.

[0413] "A means of proposing interior decoration based on a budget" refers to a function that selects furniture and decorative items that can be purchased within the budget provided by the user and proposes the most suitable interior decoration.

[0414] "Methods for generating proposals using generative AI models" refers to a function that utilizes artificial intelligence technology to automatically generate interior decoration proposals that meet the user's requests and conditions.

[0415] The system for implementing this invention starts by acquiring images of a room using the user's smartphone or camera and sending the image data to a server. The server uses image recognition technology to specify the room category from the acquired images and plans the interior decoration based on the specified category. The planned interior decoration is generated as visual information and presented to the user.

[0416] The server analyzes the user's facial expressions and voice tone using the smartphone's camera and microphone to recognize the user's emotions. Image recognition libraries (e.g., OpenCV) and emotion recognition APIs (e.g., Microsoft Azure Emotion API) are used for emotion recognition. Based on the recognized emotions, the server suggests interior decorations that correspond to those emotions.

[0417] Furthermore, based on the budget information provided by the user, the server selects interior decorations that can be purchased within the budget and provides optimal suggestions. A generative AI model (e.g., OpenAI GPT-3) is used to generate these suggestions. The generated suggestions are provided along with a link to an online platform for easy access by the user.

[0418] As a concrete example, consider a scenario where a user takes a photo of their living room with their smartphone and enters a budget of 50,000 yen. The server detects that the user has a cheerful expression and suggests bright and colorful interior decorations. The suggested items are displayed with links to online platforms.

[0419] An example of a prompt message is: "If the user's emotion is joy, please suggest bright and colorful interior items that can be purchased within a budget of 50,000 yen."

[0420] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[0421] Step 1:

[0422] The user takes a picture of the room using their smartphone camera. The captured image is sent from the device to the server. The input is the image data of the room, and the output is the image data sent to the server.

[0423] Step 2:

[0424] The server analyzes the received image data using image recognition technology (e.g., OpenCV) to specify the room category. The input is image data of the room, and the output is the specified category information. As a data processing step, features are extracted from the image and classified into categories.

[0425] Step 3:

[0426] The server plans the interior decoration based on the specified category. The planning process includes selecting appropriate furniture and decorations from a database. The input is category information, and the output is a list of the planned interior decorations.

[0427] Step 4:

[0428] The server generates visual information of the planned interior decoration. This visual information is represented as images or 3D models. The input is a list of interior decorations, and the output is the visual information. As a data operation, the selected items are visually arranged.

[0429] Step 5:

[0430] The user's emotions are recognized using their smartphone's camera and microphone. The server uses an emotion recognition API (e.g., Microsoft Azure Emotion API) to analyze the user's facial expressions and voice tone. The input is the user's facial expression data and voice data, and the output is the recognized emotion information.

[0431] Step 6:

[0432] The server suggests interior decorations that correspond to the recognized emotions. The input is emotional information, and the output is a suggestion for interior decorations that match those emotions. As a data processing step, a style appropriate to the emotion is selected.

[0433] Step 7:

[0434] The user enters budget information. The server selects interior decorations that can be purchased within the user's budget and provides optimal suggestions. The input is budget information, and the output is a suggestion of interior decorations that fit the budget. As part of the data calculation, items are selected considering price information.

[0435] Step 8:

[0436] The server generates the final proposal using a generative AI model (e.g., OpenAI GPT-3). The input consists of category information, sentiment information, and budget information, and the output is a final interior design proposal. Prompts are used to guide the AI ​​model in generating the proposal.

[0437] Step 9:

[0438] The server presents the generated proposals to the user and provides a link to the online platform. The input is the final interior design proposal, and the output is the display of the proposals to the user and a purchase link. Specifically, the server sends the proposal content to the user's device.

[0439] (Other examples)

[0440] Next, other embodiments will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0441] In modern interior design, it is difficult for users to find the optimal design from a vast array of options when selecting interior finishes for their space. Furthermore, it is challenging to quickly obtain design proposals that fit within a budget or information on available products. As a result, users often waste time and effort, and frequently do not achieve satisfactory results.

[0442] The identification processing performed by the identification processing unit 290 of the data processing device 12 in other embodiments is realized by the following means.

[0443] In this invention, the server includes means for acquiring an image of a space using a camera, means for analyzing the acquired image and determining the category of the space, and means for generating prompts to instruct the system to plan an interior design that fits the determined category, and inputting these prompts into a generating AI model. This allows the user to quickly obtain an interior design that is optimal for their space.

[0444] A "photography device" is a device used to acquire images of a space, and includes devices such as digital cameras and smartphone cameras.

[0445] "Image analysis software" is a program that analyzes acquired images and extracts features from them, and can utilize libraries such as OpenCV.

[0446] A "space category" is a classification that indicates the type and use of a space determined by image analysis, and includes the names of specific spaces such as living rooms and kitchens.

[0447] A "prompt" is a text input used to give specific instructions to a generative AI model, and is a sentence containing specific requirements for planning the interior design.

[0448] A "generative AI model" is an artificial intelligence model that generates a visual representation of interior design based on input prompts, and utilizes advanced natural language processing technologies such as OpenAI GPT-3.

[0449] "Visual representation" refers to the specific visual image of the interior design generated by the generative AI model, which is visualized using 3D rendering software.

[0450] "Communication means" refers to network technology for transmitting generated visual representations to a user's terminal, and includes data transmission via the Internet.

[0451] "Product information" refers to details of purchasable products corresponding to elements within a visual representation, including links to online platforms.

[0452] This invention is a system that enables users to quickly obtain the optimal interior design for their space. The processing of this system's program is described below in natural language.

[0453] The server first uses a digital camera or smartphone camera as an imaging device to acquire an image of the space specified by the user. This image is then analyzed using OpenCV, an image analysis software. During the analysis process, features of furniture and decorations within the image are extracted, and the category of the space is determined. For example, categories such as living room or kitchen are possible.

[0454] Next, the server generates a prompt message to instruct the system to plan the interior design based on the determined category. This prompt message is then input into the OpenAI GPT-3 generation AI model. A specific example of a prompt message would be, "Please suggest a modern interior design suitable for a living room."

[0455] The AI ​​model generates a visual representation of the interior design based on the input prompt text. This visual representation is visualized using Blender, a 3D rendering software. The generated visual representation is sent from the server to the user's terminal and displayed on the user's terminal. The user can then review it and view the design details.

[0456] Furthermore, the server analyzes elements within the generated visual representation to provide links to online platforms where the relevant products can be purchased. This analysis includes matching against a product database. For example, based on the design of a sofa included in the visual representation, it generates links to where similar sofas can be purchased.

[0457] Finally, the user inputs budget information into the server. The server generates a prompt message containing the budget information and inputs it back into the AI ​​model. An example of a prompt message might be, "Please suggest interior design options for a living room that can be purchased for under 100,000 yen." The AI ​​model generates design proposals based on the budget and presents them to the user via the server.

[0458] In this way, users can quickly obtain the interior design that best suits their space.

[0459] The flow of a specific process in another embodiment will be explained using Figure 23.

[0460] Step 1:

[0461] The server uses a digital camera or smartphone camera as a capture device to acquire images of a space specified by the user. These images are the data input to the server. The server uses OpenCV, an image analysis software, to extract features of furniture and decorations within the images. This analysis determines the category of the space (e.g., living room, kitchen). The output is the category information of the space.

[0462] Step 2:

[0463] The server generates prompt statements to instruct the system to plan the interior design based on the spatial category determined in Step 1. These prompt statements are input to the generative AI model, OpenAI GPT-3. A specific example of a prompt statement is, "Please suggest a modern interior design suitable for a living room." The output is the prompt statement that is input to the generative AI model.

[0464] Step 3:

[0465] The generating AI model generates a visual representation of the interior design based on the prompt text entered in step 2. The server visualizes the generated design data using Blender, a 3D rendering software. The output is 3D design data as a visual representation.

[0466] Step 4:

[0467] The server sends the visual representation generated in step 3 to the user's terminal. The terminal displays the received visual representation on its user interface. The user can then review it and view the design details. The output is a visual representation that the user can view.

[0468] Step 5:

[0469] Based on the visual representation generated in step 3, the server analyzes the elements within the visual representation to provide links to online platforms where the relevant products can be purchased. Specifically, it identifies furniture and decorative items included in the visual representation and retrieves the corresponding product information. The output consists of product information and purchase links.

[0470] Step 6:

[0471] The user inputs budget information into the server. The server generates a prompt message containing the budget information and inputs it back into the AI ​​model. An example of a prompt message is, "Please propose interior design ideas for a living room that can be purchased for under 100,000 yen." The AI ​​model generates design proposals based on the budget and proposes them to the user via the server. The output is an interior design proposal that fits the budget.

[0472] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0473] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0474] Other examples of generative AI include Gemini® (registered trademark) (Internet search). <url: https: gemini.google.com ?hl="ja">) are some examples.

[0475] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0476] [Second Embodiment]

[0477] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0478] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0479] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0480] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0481] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0482] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0483] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0484] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0485] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0486] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0487] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0488] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0489] "Example of form 1"

[0490] One embodiment of the present invention provides a system that includes a camera module for taking pictures of a room, an interior planning module for specifying a genre based on the taken pictures and planning an interior that matches the specified genre, and an image generation module for generating image photos of the planned interior.

[0491] "Example of form 2"

[0492] As another embodiment of the present invention, a system is provided that further includes a site guidance module that guides users to a site where the generated interior image can be purchased. Specifically, it provides a link to an online shopping site where the interior item can be purchased, based on information about the interior item included in the generated image.

[0493] "Example of form 3"

[0494] In yet another embodiment of the present invention, a system is provided that further includes a coordination suggestion module that proposes an interior design that fits the budget provided by the user. Specifically, based on the budget provided by the user, the system selects interior items that can be purchased within that budget and proposes a coordination that combines them.

[0495] The following describes the processing flow for each example form.

[0496] "Example of form 1"

[0497] Step 1: The user takes a picture of the room. This is done using a camera module built into the system.

[0498] Step 2: Based on the photos taken, the user specifies the interior design genre. The genre is specified on the system's user interface.

[0499] Step 3: Plan the interior design to match the specified genre. This planning is done automatically by the system's interior planning module.

[0500] Step 4: Generate image photos of the planned interior. This generation is performed automatically by the system's image generation module.

[0501] "Example of form 2"

[0502] Step 1: Based on the generated interior image photos, the system redirects users to websites where the corresponding interior items can be purchased. This redirection is performed automatically by the system's site redirection module.

[0503] Step 2: Specifically, based on the information about the interior items included in the generated image, provide links to online shopping sites where those items can be purchased. (Example 3)

[0504] Step 1: Based on the budget provided by the user, the system proposes an interior design coordinated to fit that budget. This proposal is automatically generated by the system's coordination proposal module.

[0505] Step 2: Specifically, based on the budget provided by the user, select interior items that can be purchased within that budget and propose a coordinated look using those items.

[0506] (Example 1)

[0507] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0508] In modern living spaces, efficiently proposing interior designs that meet individual preferences and budgets is difficult. Traditional methods often require expert advice and are time-consuming and costly. Furthermore, visualizing interior designs is challenging, making it difficult for clients to form concrete images.

[0509] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0510] In this invention, the server includes means for acquiring an image of a space using a camera; means for analyzing the acquired image and determining the category of the space based on the features in the image; means for planning an interior design that fits the determined category; means for generating a visual representation of the planned interior design using a generative AI model; and means for transmitting the generated visual representation to the user's device. This allows the user to quickly and efficiently visualize interior designs that suit their preferences and budget, and to have a concrete image of them.

[0511] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and smartphones.

[0512] A "spatial image" is digital image data that contains visual information about a specific place or room.

[0513] "Image analysis" is a technique that processes acquired image data to identify features and elements within the image.

[0514] A "spatial category" is a classification that indicates the style or theme of a space, determined based on image analysis.

[0515] "Interior design" is the process of planning the placement of furniture and decorative items to suit a particular space.

[0516] A "generative AI model" is an algorithm that uses artificial intelligence technology to generate new visual representations from data.

[0517] "Visual representation" refers to images or illustrations that visually show the results of the interior design.

[0518] "User's device" refers to a device used to receive and display the generated visual representation, and includes smartphones, computers, and other similar devices.

[0519] An "online platform" is a website or application that provides goods and services via the internet.

[0520] "Budget information" refers to data about the amount of money users can spend on interior design.

[0521] This invention is a system that allows a user to take a photograph of a room and then proposes an interior design based on that photograph. The user uses a terminal's camera to acquire an image of the room. The acquired image is sent from the terminal to a server. The server uses image analysis software to analyze the features in the image and determine the category of the space. OpenCV, a Python image processing library, can be used for this analysis.

[0522] Next, the server plans the interior design based on the determined category. This involves referencing templates in the database and selecting a design that fits the category. The server then uses a generative AI model to generate a visual representation of the planned interior design. Generative AI models such as DALL-E and Stable Diffusion can be used for this generation.

[0523] The generated visual representation is sent from the server to the user's terminal. The user can view the visual representation on their terminal and use it as a reference for interior design. For example, if the user takes a picture of their living room, the server will determine the category to be "modern" and suggest a modern interior design. The generation AI model can generate an appropriate visual representation by inputting a prompt such as, "Generate a modern living room interior design."

[0524] This system allows users to quickly and efficiently visualize interior designs that suit their preferences and budget, giving them a concrete image of what they want.

[0525] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0526] Step 1:

[0527] The user takes an image of the room using the device's camera. The user opens the smartphone's camera app and takes a picture so that the entire room is visible. The captured image is sent from the device to the server. The input is the photo of the room, and the output is the image data sent to the server.

[0528] Step 2:

[0529] The server analyzes the received image data. Using image analysis software, it identifies features within the image and determines the spatial category. OpenCV, a Python image processing library, can be used for this analysis. The input is the image data sent to the server, and the output is the determined spatial category.

[0530] Step 3:

[0531] The server plans the interior design based on the determined category. It refers to templates in the database and selects a design that fits the category. The input is the category of the space, and the output is the planned interior design.

[0532] Step 4:

[0533] The server uses a generative AI model to generate a visual representation of the planned interior design. DALL-E and Stable Diffusion can be used as the generative AI model. The prompt "Generate a modern living room interior design" is entered, and the visual representation is generated. The input is the planned interior design and the prompt, and the output is the generated visual representation.

[0534] Step 5:

[0535] The server sends the generated visual representation to the user's terminal. The user can then view the visual representation on their terminal and use it as a reference for interior design. The input is the generated visual representation, and the output is the visual representation displayed on the user's terminal.

[0536] (Application Example 1)

[0537] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0538] Modern consumers face the challenge of finding the perfect interior design for their living space from a vast array of options. Furthermore, online purchases without in-store inspection can lead to discrepancies between the image and the actual product. Additionally, receiving optimal interior design suggestions that fit within one's budget can be difficult.

[0539] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0540] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations that match the specified category, and means for generating visual information of the planned interior decorations. This allows consumers to visually confirm interiors that suit their own rooms, actually check them in stores, and receive optimal suggestions that fit their budget.

[0541] "Means for acquiring images of a room" refers to a device or method for a user to photograph their living space and import that image data into the system.

[0542] "Means for specifying a category and planning interior decoration to match the specified category" refers to an apparatus or method for analyzing acquired image data, selecting an appropriate interior style based on the user's preferences and the characteristics of the room, and creating a decoration plan that matches that style.

[0543] "Means for generating visual information of planned interior decoration" refers to an apparatus or method for generating an image of the interior in a form that can be visually confirmed by the user, based on the selected interior style.

[0544] "Means for proposing goods" refers to a device or method for proposing specific products or decorative items to a user based on a generated interior image.

[0545] "Means of guiding users to a place where they can actually see the product" refers to a device or method for guiding users to a store or exhibition space so that they can actually see the proposed product or decorative item.

[0546] "Means of providing purchasable information" refers to a device or method that provides users with information to purchase proposed goods or decorative items.

[0547] "A means of proposing interior decoration that matches the presented funds based on the presented funds" refers to a device or method for proposing the optimal interior plan within the budget presented by the user.

[0548] A description of the embodiment for carrying out the invention will be given.

[0549] The system that realizes this invention mainly consists of a user terminal and a server. The user terminal is a mobile information terminal such as a smartphone or tablet, and is equipped with a camera module for acquiring images of the room. The user uses this terminal to take pictures of their room.

[0550] The captured image data is sent from the terminal to the server. The server uses image recognition software (e.g., TensorFlow) to analyze the image data and extract the characteristics of the room. Based on this analysis, the server uses a generative AI model (e.g., OpenAI's DALL-E) to select an interior style that suits the user's preferences and the characteristics of the room, and generates visual information of the interior decoration that matches that style.

[0551] The generated visual information is sent to the user's device, where the user can visually confirm it. Furthermore, the server suggests specific products and decorative items to the user based on the generated interior image. These suggestions also include information directing the user to stores or exhibition spaces where they can actually see the products.

[0552] For example, if a user requests a "modern living room," the AI ​​model will receive a prompt such as, "Generate interior design suggestions for a modern living room. The room photos are below." This allows the user to visually see interior designs that suit their room and then actually check them out in a store. The system will also propose an optimal interior design plan based on the user's stated budget.

[0553] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0554] Step 1:

[0555] The user takes a picture of the room using the device's camera module. The input is the image data of the room taken by the user. The output is the captured image data being saved to the device.

[0556] Step 2:

[0557] The device sends the captured image data to the server. The input is the image data stored on the device. The output is the image data transferred to the server.

[0558] Step 3:

[0559] The server analyzes the received image data using image recognition software (e.g., TensorFlow). The input is the image data transferred to the server. Feature extraction is performed as part of the data processing. The output is data indicating the characteristics of the room.

[0560] Step 4:

[0561] The server uses a generative AI model (e.g., OpenAI's DALL-E) to select an interior style that matches the user's preferences and the characteristics of the room, based on room feature data. The input is room feature data. The data calculation involves selecting an interior style. The output is data of the selected interior style.

[0562] Step 5:

[0563] The server generates visual information of the interior decoration based on the selected interior style. The input is data of the selected interior style. As a data processing step, visual information is generated. The output is the generated visual information.

[0564] Step 6:

[0565] The server sends the generated visual information to the user's terminal. The input is the generated visual information. The output is the visual information transferred to the user's terminal.

[0566] Step 7:

[0567] The user reviews the visual information received on the device and visually evaluates the proposed interior. The input is the visual information displayed on the device. The output is the user's evaluation.

[0568] Step 8:

[0569] The server suggests specific products and decorations to the user based on the generated interior image and provides information guiding them to stores or exhibition spaces where they can actually view the products. The input is the generated interior image. The data calculation involves generating product suggestions and guidance information. The output is the suggested product information and guidance information.

[0570] (Example 2)

[0571] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0572] Traditional interior design systems have faced challenges such as difficulty in receiving specific interior design proposals based on the user's desired style and theme, and a lack of information for actually purchasing the suggested interior items. Furthermore, providing optimal interior coordination within the user's budget has also been difficult.

[0573] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0574] In this invention, the server includes means for acquiring images, means for specifying categories based on the acquired images and planning spatial decorations suitable for the specified categories, means for generating visual information of the planned spatial decorations, means for analyzing items included in the generated visual information, and means for providing a connection to an information source from which the analyzed items can be obtained. This allows the user to receive interior design suggestions based on their desired style and to easily purchase the suggested items. It is also possible to provide optimal interior coordination according to the user's budget.

[0575] "Means for acquiring images" refers to a device or method for collecting visual information provided by a user in digital format.

[0576] "Means for specifying categories" refers to an apparatus or method for determining a classification related to a specific style or theme based on acquired images.

[0577] "Means for planning spatial decoration" refers to an apparatus or method for constructing an interior design suitable for a specified category.

[0578] "Means for generating visual information" refers to an apparatus or method for creating a planned spatial decoration as a digital image.

[0579] "Means for analyzing articles" refers to a device or method for identifying interior items contained in generated visual information and extracting their characteristics.

[0580] "Means of providing access to information sources" refers to a device or method for generating and providing to a user a link to an online platform where the analyzed items can be purchased.

[0581] The following systems are conceivable as embodiments for carrying out this invention.

[0582] The user uses a terminal to enter prompts to generate interior design images. These prompts can specify the style or theme the user desires. For example, they might enter prompts such as "modern living room" or "Nordic-style bedroom."

[0583] The terminal sends the prompt message received from the user to the server. The server uses a generative AI model to generate an image of the interior based on the prompt message. This generative AI model can use image generation technologies such as DALL-E or Midjourney.

[0584] The generated images are analyzed by a server. The server uses image recognition technology to identify interior items contained in the images and extract their characteristics. This analysis reveals what the identified items are.

[0585] Next, the server provides a connection to information sources where the identified interior items can be purchased, based on the analysis results. Specifically, it generates and provides links to online shopping platforms. Database search technology can be used to generate these links. For example, it could provide links to platforms such as Amazon or IKEA.

[0586] Users can view images and purchase links generated through their devices. By clicking on the link for an item they are interested in, users can easily access the purchase page and actually buy the item.

[0587] This system allows users to obtain ideas for their desired interior design while simultaneously facilitating the process of actually purchasing those items.

[0588] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0589] Step 1:

[0590] The user uses a terminal to input prompt text for interior design. This prompt text specifically expresses the user's desired style or theme. For example, it might be something like "vintage-style dining room." This prompt text serves as the basic data for subsequent processing.

[0591] Step 2:

[0592] The terminal sends the prompt message received from the user to the server. The server receives this prompt message as input and requests image generation from the generative AI model. The generative AI model generates an image of the interior based on the prompt message. In this process, the content of the prompt message is analyzed and data processing is performed to generate an appropriate image. The generated image is obtained as output.

[0593] Step 3:

[0594] The server analyzes the generated image. It receives an image as input and uses image recognition technology to identify the interior items contained in the photograph. This analysis extracts the type and characteristics of the items. The output provides information about the identified items. Specifically, items such as sofas and tables are identified from the photograph.

[0595] Step 4:

[0596] The server provides a connection to information sources where the identified interior items can be purchased, based on the analysis results. It receives item information as input and uses database search technology to generate links to online shopping platforms. The output is a purchase link. Specifically, links to platforms like Amazon and IKEA are generated.

[0597] Step 5:

[0598] The terminal displays the generated image and purchase link received from the server to the user. The user can view the displayed image and click the link for an item of interest to access the purchase page. It receives the generated image and purchase link as input and provides visual information to the user as output. Specifically, the user can click the link and proceed with the purchase process.

[0599] (Application Example 2)

[0600] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0601] Modern consumers want to easily find and purchase interior decorations that suit their living spaces. However, in reality, finding the right items often requires a lot of time and effort. Furthermore, the need to compare multiple online platforms to find available items makes efficient purchasing difficult.

[0602] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0603] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations suitable for the specified category, means for generating visual information of the planned interior decorations, and means for identifying items included in the generated visual information and providing connection information to an online platform where the items can be acquired. This makes it possible for users to easily find and efficiently purchase interior decorations that suit their rooms.

[0604] "Means for acquiring images of a room" refers to a function that allows users to take pictures of a room using a mobile device and import that image data into the system.

[0605] "A means of specifying a category and planning interior decoration suitable for that category" refers to a function that selects an interior decoration style or theme based on acquired images and creates a decoration plan that matches it.

[0606] "Means for generating visual information for interior decoration" refers to a function that generates images and graphics to visually represent the planned interior decoration.

[0607] "Means for identifying items and providing connection information to online platforms where those items can be obtained" refers to a function that identifies individual items included in the generated visual information and provides users with links and information to online platforms where those items can be purchased.

[0608] The system for implementing this invention begins with a user acquiring an image of a room using a mobile device and sending that image to a server. The server uses image recognition software (e.g., TensorFlow, OpenCV) to analyze the objects in the image and specify their categories. Based on the specified categories, it uses a generative AI model (e.g., OpenAI's GPT model) to create a suitable interior decoration plan.

[0609] Next, the server generates visual information to visually represent the planned interior decoration. This visual information includes interior items that are suitable for the user's room. The server identifies each item included in the generated visual information and provides connection information to the online platform where they can be obtained. This allows the user to easily purchase items they are interested in.

[0610] As a concrete example, when a user takes a picture of their living room and sends it to the system, the server recognizes items such as a sofa, table, and lamp. For each item, it generates links to online platforms such as Amazon and Rakuten Market and provides them to the user. By clicking on the provided links, the user can directly access the purchase page for the corresponding item.

[0611] An example of a prompt would be, "Please tell me where I can buy the same sofa shown in this picture." By inputting this prompt into the AI ​​model, a link to the appropriate online platform will be generated.

[0612] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0613] Step 1:

[0614] The user takes a picture of the room using a mobile device. The captured image is saved on the device.

[0615] Step 2:

[0616] The terminal sends the captured image to the server. The input is image data of the room, and the output is the transfer of the image data to the server.

[0617] Step 3:

[0618] The server analyzes the received image data using image recognition software (e.g., TensorFlow, OpenCV). The input is image data, and the output is category information of the objects in the image. The server identifies the objects in the image and assigns each category to them.

[0619] Step 4:

[0620] The server uses a generative AI model to create interior decoration plans based on specified categories. The input is category information for items, and the output is an interior decoration plan. The server generates decoration ideas suitable for each category.

[0621] Step 5:

[0622] The server generates visual information to visually represent the planned interior decoration. The input is the interior decoration plan, and the output is visual information (image data). The server generates images to visualize the decoration plan.

[0623] Step 6:

[0624] The server identifies each item included in the generated visual information and provides connection information to the online platform where they can be obtained. The input is visual information, and the output is link information to the online platform. The server generates a purchase link for each item.

[0625] Step 7:

[0626] Users receive link information provided through their device and select items they are interested in. By clicking on the link, users can access the purchase page for the selected item.

[0627] (Example 3)

[0628] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0629] Traditional interior design systems struggled to provide optimal suggestions tailored to the user's budget, and verifying whether the proposed designs were actually affordable was cumbersome. Furthermore, there was a lack of effective means to utilize user feedback to improve the system.

[0630] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0631] In this invention, the server includes means for acquiring images, means for specifying categories based on the acquired images and planning decorative items suitable for the specified categories, means for generating visual information of the planned decorative items, means for receiving budget information from the user, selecting purchasable decorative items based on the budget information and generating suggestions combining the selected decorative items, and means for suggesting the optimal combination of the selected decorative items using a generation AI model. This makes it possible to suggest the optimal interior coordination according to the user's budget, and also makes it easier to confirm that the suggested coordination is actually purchasable. Furthermore, user feedback can be received and used to improve the system.

[0632] "Means for acquiring images" refers to a device or method for visually recording the user's environment or the state of a room.

[0633] "Means of specifying categories" refers to methods for classifying interior styles and themes based on acquired images.

[0634] "Means of planning decorative items" refers to methods for selecting interior items suitable for a specified category and considering their placement.

[0635] "Means of generating visual information" refers to methods for visually representing the planned layout and design of an interior.

[0636] "Means of receiving budget information" refers to methods for obtaining information about the amount of money that users are willing to spend.

[0637] "Methods for selecting decorative items" refers to methods for selecting interior items that are affordable within the user's budget.

[0638] "Means for generating proposals" refers to a method for combining selected decorative items to propose the optimal interior design for the user.

[0639] "Methods using generative AI models" refer to methods that utilize artificial intelligence technology to calculate and propose the optimal combination of selected decorative items.

[0640] "Means of receiving feedback" refers to methods for obtaining opinions and evaluations from users and using them to improve the system.

[0641] A description of embodiments for carrying out this invention will be given.

[0642] The user first accesses the interface using a terminal and enters budget information. The terminal sends this input to the server. The server then searches its database based on the budget information received from the user. Specifically, it uses a database management system such as MySQL to query for interior items that can be purchased within the budget.

[0643] The server generates a coordinated look by combining interior items selected from the search results. This process can utilize a generative AI model. Specifically, the AI ​​model is given a prompt message such as, "Please suggest the best coordinated look using the selected items within the budget," and the AI ​​generates the suggestions.

[0644] The generated outfit suggestions are sent from the server to the terminal. The terminal displays the suggestions to the user. The user can review the suggested outfits and provide feedback as needed. The terminal sends this feedback to the server to help improve the system.

[0645] As a concrete example, if a user specifies a budget of "50,000 yen," the system will select items such as a sofa, table, and lamp that can be purchased within that budget and propose a coordinated look combining them. In this way, the user can receive interior design suggestions that match their budget. The specific processing flow in Example 3 will be explained using Figure 15.

[0646] Step 1:

[0647] The user accesses the interface using their device and enters budget information. Specifically, the user enters the budget amount into a web form or app input field and clicks the "Submit" button. The entered budget information is sent from the device to the server.

[0648] Step 2:

[0649] The server searches the database based on the budget information received from the user. Specifically, it uses a database management system such as MySQL to query for interior items that can be purchased within the budget. The input is the budget information, and the output is a list of items that can be purchased within the budget. The server considers the price information of the items and selects only those items that do not exceed the budget.

[0650] Step 3:

[0651] The server generates a coordinated interior design by combining selected interior items from the search results. A generative AI model can be used for this process. Specifically, the AI ​​model is given the prompt, "Please suggest the best coordinated design using items selected within the budget," and the AI ​​generates the suggestion. The input is a list of items, and the output is the coordinated design suggestion.

[0652] Step 4:

[0653] The server sends the generated outfit suggestions to the terminal. The terminal displays the suggestions to the user. Specifically, the terminal displays images and descriptions of the outfits on a web page or app screen, allowing the user to review them. The input is the outfit suggestions, and the output is the visual display to the user.

[0654] Step 5:

[0655] Users can review suggested outfits and provide feedback. The device sends this feedback to the server to help improve the system. Specifically, users can choose options such as "I like this outfit" or "I want to see other suggestions." The input is user feedback, and the output is information related to system improvements.

[0656] (Application Example 3)

[0657] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0658] Modern consumers face the challenge of achieving optimal interior design within a limited budget. Furthermore, selecting and combining interior items requires specialized knowledge, making it a high hurdle for the average consumer. Therefore, there is a need for a system that can easily propose optimal interior design solutions within a given budget.

[0659] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0660] In this invention, the server includes means for acquiring images of a space using a camera; means for selecting a category based on the acquired images and planning decorative items that fit the selected category; and means for receiving budget information from the user, selecting decorative items that can be purchased within that budget, and proposing the optimal combination. This makes it possible for consumers to easily achieve the optimal interior design within their budget.

[0661] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and smartphones.

[0662] "Spatial images" are photographs or image data that contain visual information about a room or a specific area.

[0663] A "category" is a standard used to classify interior items, and is selected based on style and purpose.

[0664] "Decorative items" refer to furniture, accessories, and other items used in interior design.

[0665] "Planning" is the process of creating an interior design by combining decorative items that fit the selected category.

[0666] A "visualized image" is an image that visually represents the planned arrangement and combination of decorative items.

[0667] "Budget information" refers to data about the amount of money users can spend on interior design.

[0668] An "online platform" refers to a website or application that allows users to purchase goods over the internet.

[0669] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and select decorative items that can be purchased within a budget.

[0670] The system for implementing this invention mainly consists of a server and a user terminal. The server has the function of acquiring images of a space using a camera and receives image data transmitted from the user terminal. Based on the acquired images, the server selects a category and processes items to plan decorative items that fit the selected category. In this process, the server uses a generative AI model to consider budget information provided by the user and selects decorative items that can be purchased within the budget.

[0671] The server generates a visualization of the planned decorations and sends it to the user's terminal. The user can view the visualization on their terminal and purchase the suggested decorations through the online platform. The server generates and provides a link to the online platform to the user.

[0672] As a concrete example, if a user takes a picture of their living room with their smartphone and enters a budget of 50,000 yen, the server analyzes the image and selects a category suitable for the living room. The generating AI model then selects decorative items such as sofas, tables, and curtains that can be purchased within 50,000 yen and suggests the optimal combination. The user can then review the suggested coordination and purchase the items they like.

[0673] An example of a prompt for a generating AI model is, "Please select interior items that can be purchased for a budget of 50,000 yen and suggest a living room coordination." This prompt allows the AI ​​model to provide optimal suggestions tailored to the user's needs.

[0674] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[0675] Step 1:

[0676] The user takes a picture of the room using their smartphone and sends it to the server through the application. The input is the image data of the room, and the output is the transfer of the image data to the server. In this step, the smartphone's camera function is used to acquire the image and upload it to the server via the internet.

[0677] Step 2:

[0678] The server analyzes the received image data and uses image recognition technology to select a room category. The input is image data sent by the user, and the output is the selected category information. The server uses an image analysis algorithm to extract room features and determine the appropriate category.

[0679] Step 3:

[0680] The user enters budget information through the application and sends it to the server. The input is the budget information specified by the user, and the output is the transfer of budget data to the server. In this step, the user specifies the budget using the application's input form and sends it to the server.

[0681] Step 4:

[0682] The server uses a generative AI model to select purchasable decorative items based on the chosen category and budget information. The input is category information and budget information, and the output is a list of selected decorative items. The server inputs prompts into the AI ​​model to select the most suitable decorative item within the budget.

[0683] Step 5:

[0684] The server generates a visualization image based on the selected ornaments and sends it to the user terminal. The input is a list of ornaments, and the output is a visualization image. The server uses a graphics generation algorithm to visualize the arrangement of the ornaments and generate the image.

[0685] Step 6:

[0686] The user views visualized images on their device and selects their preferred decorative items. The input is the visualized images, and the output is the user's selection information. The user uses the application interface to review the suggested decorative items and select the items they wish to purchase.

[0687] Step 7:

[0688] The server generates and provides to the user a link to the online platform corresponding to the selected decorative item. The input is the user's selection information, and the output is a link to the online platform. The server generates a link to the purchase page for the selected item and sends it to the user.

[0689] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0690] "Example of form 1"

[0691] One embodiment of the present invention is a system that combines a user emotion engine with a system that recognizes the user's emotions. This system takes a photo of a room and specifies a genre based on the photo. It plans an interior that matches the specified genre and generates an image of the planned interior. Furthermore, based on the generated image of the interior, it guides the user to a website where the corresponding interior can be purchased. Finally, based on the budget provided by the user, it proposes an interior coordination that fits the budget. In addition to this series of steps, a user emotion engine is incorporated to recognize the user's emotions. Specifically, it estimates the user's emotions from their facial expressions, tone of voice, and speed of operation when they operate the system, and proposes an interior that matches that emotion. For example, if the user is showing feelings of joy, it will propose an interior with a bright and cheerful atmosphere. Conversely, if the user is feeling down, it will propose a calm and soothing interior. This makes it possible to provide more personalized interior proposals that respond to the user's emotions.

[0692] "Example of form 2"

[0693] One embodiment of the present invention is a system that combines a user emotion engine with a system that recognizes the user's emotions. This system takes a photo of a room and specifies a genre based on the photo. It plans an interior that matches the specified genre and generates an image of the planned interior. Furthermore, based on the generated image of the interior, it guides the user to a website where the corresponding interior can be purchased. Finally, based on the budget provided by the user, it proposes an interior coordination that fits the budget. In addition to this series of steps, a user emotion engine is incorporated to recognize the user's emotions. Specifically, it estimates the user's emotions from their facial expressions, tone of voice, and speed of operation when they operate the system, and proposes an interior that matches that emotion. For example, if the user is showing feelings of joy, it will propose an interior with a bright and cheerful atmosphere. Conversely, if the user is feeling down, it will propose a calm and soothing interior. This makes it possible to provide more personalized interior proposals that respond to the user's emotions.

[0694] "Example of form 3"

[0695] One embodiment of the present invention is a system that combines a user emotion engine with a system that recognizes the user's emotions. This system takes a photo of a room and specifies a genre based on the photo. It plans an interior that matches the specified genre and generates an image of the planned interior. Furthermore, based on the generated image of the interior, it guides the user to a website where the corresponding interior can be purchased. Finally, based on the budget provided by the user, it proposes an interior coordination that fits the budget. In addition to this series of steps, a user emotion engine is incorporated to recognize the user's emotions. Specifically, it estimates the user's emotions from their facial expressions, tone of voice, and speed of operation when they operate the system, and proposes an interior that matches that emotion. For example, if the user is showing feelings of joy, it will propose an interior with a bright and cheerful atmosphere. Conversely, if the user is feeling down, it will propose a calm and soothing interior. This makes it possible to provide more personalized interior proposals that respond to the user's emotions.

[0696] The following describes the processing flow for each example form.

[0697] "Example of form 1"

[0698] Step 1: The user takes a photo of the room.

[0699] Step 2: The system specifies a genre based on the photos it has taken.

[0700] Step 3: The system plans an interior design that matches the specified genre and generates image photos of the planned interior.

[0701] Step 4: Based on the interior image photos generated by the system, users are directed to a website where the corresponding interior items can be purchased.

[0702] Step 5: Based on the budget provided by the user, the system proposes interior design ideas that fit within that budget.

[0703] Step 6: The system uses an emotion engine to recognize the user's emotions, estimating their feelings from their facial expressions, tone of voice, and speed of operation while they interact with the system.

[0704] Step 7: The system suggests interior design elements that match the user's estimated emotions. For example, if the user is expressing joy, the system suggests a bright and cheerful interior design. Conversely, if the user is feeling down, the system suggests a calming and soothing interior design.

[0705] (Example 1)

[0706] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0707] In modern living spaces, there is a demand for personalized interior design proposals that cater to the individual user's emotions and budget. However, conventional systems struggle to provide interior design proposals that take user emotions into account, and they also have difficulty quickly providing optimal proposals that fit within the budget.

[0708] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0709] In this invention, the server includes means for acquiring images of a space using a camera; means for identifying categories based on the acquired images and planning interiors that fit the identified categories; means for generating visual information of the planned interiors; and means for recognizing the user's emotions using an emotion analysis device and adjusting interior design suggestions based on the recognized emotions. This enables personalized interior design suggestions that are tailored to the user's emotions and budget.

[0710] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and sensors.

[0711] A "spatial image" is digital image data that includes visual information of a room or a specific location.

[0712] "Category" refers to the genre or style of interior design classified based on the characteristics of the space.

[0713] "Interior design" refers to all aspects of design, including the decoration of rooms and spaces, and the arrangement of furniture.

[0714] "Planning methods" refer to the processes and methods for designing and proposing interiors that fit a specific category.

[0715] "Visual information" refers to images and illustrations that visually represent the planned interior design.

[0716] An "emotion analysis device" is a device designed to recognize the user's emotions and has the function of analyzing facial expressions, tone of voice, operation speed, etc.

[0717] "User emotions" refers to the psychological state or feelings that users exhibit when operating the system.

[0718] "Means of adjusting proposals" refers to methods or processes for modifying or optimizing interior design proposals based on perceived emotions.

[0719] The following system is constructed as an embodiment of this invention.

[0720] The server first receives an image of the space transmitted from the terminal. This image is taken by the user using a camera. The server uses a generative AI model to analyze the received image and extract spatial features from it. Based on these features, the server identifies a category and plans an interior design that fits that category.

[0721] The interior planning module is used for interior design planning. This module searches the database for relevant interior items and generates a list of furniture and decorations that fit the categories. The server then uses the image generation module to generate visual information of the planned interior. This visual information is intended to provide the user with a concrete image of the interior.

[0722] Furthermore, the server uses an emotion analysis device to recognize the user's emotions. It analyzes data such as the user's facial expressions, tone of voice, and operation speed to estimate the user's emotions. Based on this emotional information, the server adjusts its interior design suggestions. For example, if the user is showing an emotion of joy, the server sends a prompt message to the AI ​​model saying, "Please suggest an interior with a bright and cheerful atmosphere," and then makes a suggestion.

[0723] As a concrete example, here is an example of a prompt message to be input into a generative AI model: "Analyze the room photos taken by the user and specify a genre. Plan an interior design that matches that genre and generate an image photo. Also, recognize the user's emotions and provide interior design suggestions that match those emotions."

[0724] In this way, the server can provide personalized interior design suggestions tailored to the user's emotions and budget.

[0725] The flow of the specific processing in Example 1 will be explained using Figure 17.

[0726] Step 1:

[0727] The user takes a picture of the room using the camera on their device. The captured image is sent from the device to the server. The input is the image data of the room, and the output is the transmission of the image data to the server.

[0728] Step 2:

[0729] The server analyzes the received image data. Using a generative AI model, it extracts spatial features from the image and identifies categories. The input is the received image data, and the output is the identified category information.

[0730] Step 3:

[0731] The server uses an interior planning module to plan interiors that fit a specified category. It searches the database for relevant interior items and generates a list of furniture and decorations that match the category. The input is category information, and the output is a list of interior items.

[0732] Step 4:

[0733] The server uses an image generation module to generate visual information of the planned interior. It sends a prompt message to the generation AI model saying, "Create an image based on this interior list," and creates a realistic image. The input is a list of interior items, and the output is visual information of the interior.

[0734] Step 5:

[0735] The server recognizes the user's emotions using an emotion analysis device. It analyzes data such as the user's facial expressions, voice tone, and operation speed to estimate their emotions. The input is the user's operation data, and the output is the estimated emotion information.

[0736] Step 6:

[0737] The server adjusts interior design suggestions based on the recognized emotional information. For example, if the user indicates a feeling of joy, the server sends a prompt message to the AI ​​model saying, "Please suggest an interior with a bright and cheerful atmosphere," and then makes a suggestion. The input is emotional information, and the output is an adjusted interior design suggestion.

[0738] Step 7:

[0739] The server uses the generated visual information of the interior design to guide users to information sources where that interior design is available for purchase. Users can view this information through their terminal and purchase items that interest them. The input is the visual information of the interior design, and the output is a guide to purchase information.

[0740] (Application Example 1)

[0741] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0742] Modern consumers are seeking personalized interior design proposals that cater to their individual emotions and budgets. However, traditional interior design proposal systems have faced challenges in considering user emotions and providing optimal proposals within a given budget.

[0743] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0744] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations that match the specified category, means for generating visual information of the planned interior decorations, and means for recognizing the user's emotions and suggesting interior decorations that correspond to those emotions. This makes it possible to suggest personalized interior decorations that correspond to the user's emotions and budget.

[0745] "Means for acquiring images of a room" refers to a device or method for photographing the interior of a room and acquiring the resulting image data.

[0746] "Means for specifying categories" refers to a device or method for classifying the style or theme of interior decoration based on acquired image data.

[0747] "Means for planning interior decoration" refers to an apparatus or method for devising the appropriate arrangement of furniture and decorative items based on a specified category.

[0748] "Means for generating visual information" refers to a device or method for visually representing the image of a planned interior decoration.

[0749] "Means for recognizing user emotions" refers to a device or method for estimating a user's emotions from their facial expressions, tone of voice, etc.

[0750] "Means for proposing interior decoration" refers to a device or method for presenting optimal interior decoration in accordance with the recognized emotions of the user.

[0751] The system for implementing this invention begins with the user acquiring images of a room using a smartphone or smart glasses. The device uses a camera module to capture images of the room and sends the data to a server. The server analyzes the acquired images using image recognition technology and specifies the category of interior decoration. Image recognition software such as the Google Cloud Vision API can be used for this purpose.

[0752] Next, the server plans the interior decoration based on the specified category. Visual information of the planned interior decoration is generated using a generative AI model. By utilizing generative AI models such as OpenAI GPT-3, it is possible to provide users with visually appealing images of interior decoration.

[0753] Furthermore, the device analyzes the user's facial expressions and tone of voice to recognize their emotions. Using technologies such as Microsoft Azure Face API and IBM Watson Speech to Text, it can estimate the user's emotions. The server then suggests optimal interior decorations based on the recognized emotions.

[0754] For example, if a user takes a picture of their living room using smart glasses, the server analyzes the image and assigns it the "modern style" category. If the server detects that the user is smiling, it suggests furniture and decorations in bright colors. An example of a prompt might be, "Suggest products that match the interior of this photo. The user's emotion is joy."

[0755] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[0756] Step 1:

[0757] The user takes pictures of the room using a smartphone or smart glasses. The input is image data acquired through the camera module, and the output is that same image data. The image is captured when the user presses the capture button.

[0758] Step 2:

[0759] The device sends the captured image data to the server. The input is the image data acquired in step 1, and the output is the image data sent to the server. The device uploads the data over the network.

[0760] Step 3:

[0761] The server analyzes the received image data using image recognition technology and specifies the category of interior decoration. The input is the image data sent to the server, and the output is the specified category. The server uses the Google Cloud Vision API to analyze the image and determine the category.

[0762] Step 4:

[0763] The server plans interior decorations based on specified categories and generates visual information using a generative AI model. The input is the specified categories, and the output is the generated visual information. The server uses OpenAI GPT-3 to generate images of visually appealing interior decorations.

[0764] Step 5:

[0765] The device analyzes the user's facial expressions and voice tone to recognize emotions. Input is the user's facial expression and voice data, and output is the recognized emotion. The device uses Microsoft Azure Face API and IBM Watson Speech to Text to estimate emotions.

[0766] Step 6:

[0767] The server suggests optimal interior design based on the recognized emotions. The input is the recognized emotions and generated visual information, while the output is the suggested interior design. The server adjusts the suggestions based on the user's emotions.

[0768] (Example 2)

[0769] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0770] Modern consumers demand personalized interior design suggestions tailored to their individual preferences and budgets, but traditional systems struggle to efficiently achieve this. There is also a need for easier access to available decorative items.

[0771] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0772] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images, means for planning interior decoration that matches the specified category, means for generating visual information of the planned interior decoration, means for analyzing information on decorative items included in the generated visual information, means for providing links to information sources where decorative items can be purchased based on the analysis results, and means for recognizing the user's emotions and suggesting interior decoration that corresponds to those emotions. This makes it possible to suggest personalized interior decoration that corresponds to the user's emotions and budget, and also makes it easier to access purchasable decorative items.

[0773] "Means for acquiring images of a room" refers to a device or method for receiving a photograph of a room taken by a user as digital data.

[0774] "Means for specifying a category" refers to a device or method for analyzing acquired images of a room and determining an appropriate interior genre or style based on those images.

[0775] "Means for planning interior decoration" refers to an apparatus or method for constructing an appropriate interior design based on a specified category.

[0776] "Means for generating visual information" refers to an apparatus or method for creating images or graphics to visually represent a planned interior decoration.

[0777] "Means for analyzing information on decorative items" refers to a device or method for identifying individual interior items included in the generated visual information and obtaining detailed information therefrom.

[0778] "Means of providing links to information sources" refers to devices or methods for providing users with links to online platforms or stores where the analyzed ornaments can be purchased.

[0779] "Means for recognizing user emotions" refers to a device or method for analyzing a user's facial expressions, tone of voice, operation speed, etc., to estimate the user's emotional state.

[0780] "Means of proposing interior decoration" refers to a device or method for presenting an optimal interior design to a user based on their perceived emotions and budget.

[0781] The following systems are conceivable as embodiments for carrying out this invention.

[0782] The server receives images of a room taken by the user using their device. The images are sent to the server as digital data, and the server analyzes the images using image recognition technology. This analysis determines the appropriate interior category based on the room's style and characteristics. For example, if the image contains a lot of wooden furniture, the "natural" category will be assigned.

[0783] Next, the server uses a generative AI model to plan the interior based on the specified category. The generative AI model is given specific instructions as prompts, such as "Please suggest an interior for a natural living room." Based on these prompts, the AI ​​model generates an interior image as visual information.

[0784] The generated interior image is sent from the server to the terminal and displayed to the user. The terminal analyzes each interior item included in the image and provides links to sources where each item can be purchased. For example, if the sofa shown in the image is available for purchase at a specific online store, a link to that store is presented to the user.

[0785] Furthermore, the device uses an emotion engine to analyze the user's facial expressions, tone of voice, and operation speed to estimate the user's emotions. Based on this emotion information, the server provides interior design suggestions that match the user's emotions. For example, if the user is expressing joy, the server will suggest an interior design with a bright and cheerful atmosphere.

[0786] In this way, users can easily find personalized interiors that suit their mood and budget.

[0787] The flow of the specific processing in Example 2 will be explained using Figure 19.

[0788] Step 1:

[0789] The user takes a picture of the room using their smartphone or camera. The captured image is saved as digital data on the device. The device then sends this image data to the server. The input is the image data of the room, and the output is the transmission of the image data to the server.

[0790] Step 2:

[0791] The server analyzes the received image data of the room. Using image recognition technology, it extracts features from the image and assigns an interior category. For example, if the image contains a lot of wooden furniture, the "natural" category will be assigned. The input is image data of the room, and the output is the assigned interior category.

[0792] Step 3:

[0793] The server uses a generative AI model to plan interiors based on a specified category. A prompt, such as "Please suggest a natural living room interior," is input to the generative AI model. The AI ​​model then generates interior images based on this prompt. The input consists of an interior category and a prompt, while the output is an interior image.

[0794] Step 4:

[0795] The server sends the generated interior image to the terminal. The terminal displays the received image to the user. The input is the interior image, and the output is the display of the image to the user.

[0796] Step 5:

[0797] The device analyzes each interior item included in the displayed interior image. Based on the analysis results, it generates and provides to the user links to sources where each item can be purchased. For example, if the sofa shown in the image is available for purchase at a specific online store, a link to that store will be displayed. The input is an interior image, and the output is a purchase link.

[0798] Step 6:

[0799] The terminal analyzes the user's facial expressions, tone of voice, and operation speed using an emotion engine to estimate the user's emotions. Based on this emotion information, the server provides interior design suggestions that correspond to the user's emotions. For example, if the user is expressing joy, it will suggest an interior design with a bright and cheerful atmosphere. The input is the user's emotion data, and the output is an interior design suggestion that corresponds to those emotions.

[0800] (Application Example 2)

[0801] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0802] Conventional interior design proposal systems have challenges in providing personalized suggestions that cater to users' emotions and budgets, and in offering insufficient means for users to easily purchase the suggested decorative items. Furthermore, because interior design suggestions are not based on users' emotions, it is difficult to improve user satisfaction.

[0803] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0804] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations suitable for the specified category, means for generating visual information of the planned interior decorations, and means for recognizing the user's emotions and suggesting interior decorations corresponding to those emotions. This makes it possible to suggest personalized interior decorations that match the user's emotions and budget, and further makes it possible to easily purchase the suggested decorative items.

[0805] "Method for acquiring images of the room" refers to a function that allows users to take photos of the room using their smartphones or cameras and import that image data into the system.

[0806] "Means for specifying a category and planning interior decoration suitable for the specified category" refers to a function that analyzes acquired images of a room, identifies the interior style and theme, and then selects and plans appropriate interior decoration based on that.

[0807] "Means for generating visual information of interior decoration" refers to a function that generates images and graphics to visually represent the planned interior decoration.

[0808] "A means of recognizing the user's emotions and proposing interior decorations that correspond to those emotions" refers to a function that analyzes the user's facial expressions and tone of voice to estimate their emotions and then proposes interior decorations that are appropriate for those emotions.

[0809] "Means of directing users to online platforms" refers to a function that provides links to online shopping sites where related decorative items can be purchased, based on the generated visual information of the interior decorations.

[0810] "A means of proposing interior decorations that fit the budget provided" refers to a function that selects and proposes interior decorations that are available for purchase within the budget range provided by the user.

[0811] The system for implementing this invention consists of a user terminal and a server. The user terminal is a portable information terminal such as a smartphone or tablet, equipped with a camera and microphone. The server provides computing resources for image processing, emotion recognition, and interior design proposals using generative AI models.

[0812] The user's device uses a camera to acquire images of the room. The acquired images are sent to a server, which analyzes the images using image recognition software (e.g., TensorFlow, OpenCV) and assigns a category. Based on the assigned category, the server uses a generative AI model (e.g., DALL-E, Stable Diffusion) to generate visual information for suitable interior decorations.

[0813] Furthermore, the user's device uses a microphone and camera to capture the user's facial expressions and voice tone, and sends this information to the server. The server uses emotion recognition software (e.g., Affectiva, Microsoft Azure Emotion API) to estimate the user's emotions and suggests interior decorations that correspond to those emotions.

[0814] The proposed interior decorations are displayed as visual information on the user's device, and links to online platforms where the related decorations can be purchased are provided. Based on the presented budget, the user can select interior decorations that are available within that budget.

[0815] For example, if a user takes a photo of their room with their smartphone and the app recognizes that the user wants to relax, the server will suggest calming color schemes for the interior and display a link to purchase the items. An example of a prompt would be, "Please suggest relaxing interior items that would suit this room."

[0816] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[0817] Step 1:

[0818] The user uses the device's camera to capture images of the room. The captured images are sent from the device to the server. The input is the image data of the room, and the output is the transmission of the image data to the server.

[0819] Step 2:

[0820] The server analyzes the received image data using image recognition software (e.g., TensorFlow, OpenCV). The analysis results in a category based on the room's style and theme. The input is image data of the room, and the output is the specified category information.

[0821] Step 3:

[0822] The server generates visual information for suitable interior decorations using a generative AI model (e.g., DALL-E, Stable Diffusion) based on the specified category information. The input is category information, and the output is visual information for interior decorations.

[0823] Step 4:

[0824] The user uses the device's microphone and camera to capture their facial expressions and voice tone, and sends this data to the server. The input is the user's facial expression and voice tone data, and the output is the data transmission to the server.

[0825] Step 5:

[0826] The server analyzes the received facial expression and voice tone data using emotion recognition software (e.g., Affectiva, Microsoft Azure Emotion API) to estimate the user's emotion. The input is facial expression and voice tone data, and the output is the estimated emotion information.

[0827] Step 6:

[0828] The server suggests interior design elements suitable for the user based on estimated emotional information. The suggested interior design elements are displayed as visual information on the terminal. The input consists of emotional information and visual information of the interior design elements, and the output is the suggested display to the user.

[0829] Step 7:

[0830] Users can view visual information about interior decorations displayed on their device and proceed with a purchase by clicking on links to online platforms where related decorations can be purchased. The input is the user's selection, and the output is access to the online platform.

[0831] (Example 3)

[0832] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0833] Modern consumers demand personalized interior design tailored to their individual emotions and budgets, but traditional systems struggle to efficiently achieve this. In particular, adjusting emotionally-based suggestions and selecting the most suitable decorative items within a given budget are difficult.

[0834] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[0835] In this invention, the server includes means for acquiring images of a space using a camera, means for determining the category of the space based on the acquired images, means for planning decorative items that fit the determined category, means for analyzing the user's emotions, and means for adjusting the decorative item suggestions based on the analyzed emotions. This makes it possible to propose personalized spatial decorations that suit the user's emotions and budget.

[0836] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and smartphones.

[0837] A "space category" is a classification that indicates the style or theme of a space, determined based on the acquired images, and refers to genres such as "modern" or "classic."

[0838] "Decorative items" are items used to decorate a space, and include furniture, interior accessories, and works of art.

[0839] "Visual representation" refers to images or illustrations that visually show the planned arrangement and combination of decorative items.

[0840] "User emotions" refer to the psychological state a user exhibits when using the system, and are estimated from factors such as facial expressions, tone of voice, and speed of operation.

[0841] "Analyzing" is the process of analyzing data to extract specific information, and is done to understand emotions or the content of images.

[0842] "Adjusting the proposal" means changing the selection and placement of decorative items to suit the user's needs based on the analyzed information.

[0843] "Information provision" means providing links to online stores or retailers where the decorative items can be obtained, based on the generated visual representation.

[0844] "Financial information" refers to information about the budget provided by the user, and serves as a constraint in the selection of decorative items.

[0845] This system allows users to take pictures of their spaces and receive suggestions for decorative items that suit their budget and preferences. Users take photos of their rooms using a terminal and upload the images to the system. The server uses image analysis software to determine the category of the space. This analysis uses image recognition technology to identify categories such as "modern" or "classic" based on the room's color scheme and furniture style, for example.

[0846] Next, the server selects appropriate decorative items from the database based on the determined category and the user's budget. The server filters the price information of the decorative items to find the best combination within the budget. In this process, a generative AI model is used to generate visual representations of the decorative items. The generative AI model receives a prompt and generates an image that meets the specified conditions.

[0847] Furthermore, the server analyzes the user's facial expressions, tone of voice, and operation speed obtained from the terminal, and uses an emotion engine to estimate the user's emotions. Based on the analyzed emotions, the server adjusts its suggestions for accessories. For example, if the user wants to relax, it will suggest accessories in calming colors.

[0848] Finally, the server presents the user with a visual representation of the generated ornament and provides a link to the source of information where the ornament can be obtained. The user can click the link through their device and proceed with the purchase.

[0849] As a concrete example, if a user inputs "My budget is 50,000 yen, and I would like a relaxing living room," the server will suggest relaxing decorative items that can be purchased for under 50,000 yen and generate an image of them using a generation AI model. An example of a prompt would be, "Please suggest relaxing living room decorative items that can be purchased for under 50,000 yen." The specific processing flow in Example 3 will be explained using Figure 21.

[0850] Step 1:

[0851] The user takes a picture of the room using their device and uploads the image to the system. The input is the image of the room taken by the user. The output is the image data sent to the server. Specifically, the user uses their smartphone's camera function to take a picture of the entire room and uploads the image through the application.

[0852] Step 2:

[0853] The server inputs the received image data into image analysis software to determine the category of the space. The input is image data sent by the user. The output is the category information of the analyzed space. Specifically, the server uses an image recognition algorithm to analyze the color tones and furniture styles within the image and identify categories such as "modern" or "classic."

[0854] Step 3:

[0855] The server takes the determined category and user budget information as input and selects appropriate decorative items from the database. The input is the spatial category information and the user's budget information. The output is a list of decorative items that can be purchased within the budget. Specifically, the server executes a database query and filters out decorative items that fit the category and are within the budget.

[0856] Step 4:

[0857] The server uses a generative AI model to generate visual representations of selected ornaments. The input is a list of selected ornaments. The output is the generated visual representation of the ornaments. Specifically, the server inputs prompt statements into the generative AI model and generates images that meet the conditions.

[0858] Step 5:

[0859] The server uses an emotion engine to analyze the user's emotions, taking as input the user's facial expressions, voice tone, and operation speed obtained from the terminal. The input consists of the user's facial expression data and voice data. The output is the analyzed user emotion information. Specifically, the server uses a machine learning model to analyze the characteristics of facial expressions and voice and estimate the user's emotions.

[0860] Step 6:

[0861] The server adjusts accessory suggestions based on the analyzed emotional information. The input is the user's emotional information and a visual representation of the accessory. The output is the adjusted accessory suggestions. Specifically, the server changes the color and style of the accessories according to the user's emotions, providing the most suitable suggestions.

[0862] Step 7:

[0863] The server presents the user with a visual representation of the generated ornament and provides links to information sources where the ornament can be obtained. The input is a suggested, customized ornament. The output is the visual representation of the ornament and link information presented to the user. Specifically, the server sends the image and link to the terminal, through which the user can proceed with the purchase process.

[0864] (Application Example 3)

[0865] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0866] Modern consumers are seeking personalized interior design suggestions that cater to their individual emotions and budgets. However, traditional systems struggle to provide suggestions that consider user emotions and offer optimal solutions within a given budget. Furthermore, there is a lack of easy ways for consumers to purchase the suggested interior design items.

[0867] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[0868] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations that match the specified category, means for generating visual information of the planned interior decorations, means for recognizing the user's emotions, means for suggesting interior decorations that correspond to the emotions based on the recognized emotions, means for suggesting interior decorations that match the budget provided by the user, and means for generating suggestions using a generative AI model. This makes it possible to suggest personalized interior decorations that correspond to the user's emotions and budget, and furthermore, makes it possible to easily purchase the suggested decorations.

[0869] "Method for acquiring images of the room" refers to a function that allows users to take photos of the room using their smartphones or cameras and import that image data into the system.

[0870] The "means for specifying categories" refer to a function that classifies the style and theme of interior decoration based on acquired images of rooms and selects the appropriate category.

[0871] "Means of planning interior decoration" refers to the function of selecting furniture and decorative items suitable for a room based on a specified category, and constructing the overall design.

[0872] "Means for generating visual information" refers to functions for visually representing the planned interior design as images or 3D models.

[0873] "Means of recognizing user emotions" refers to functions that analyze the user's facial expressions and tone of voice to estimate their emotional state.

[0874] "A means of proposing interior decoration that responds to emotions" refers to a function that selects and proposes interior decoration with an appropriate atmosphere and style based on the recognized emotions of the user.

[0875] "A means of proposing interior decoration based on a budget" refers to a function that selects furniture and decorative items that can be purchased within the budget provided by the user and proposes the most suitable interior decoration.

[0876] "Methods for generating proposals using generative AI models" refers to a function that utilizes artificial intelligence technology to automatically generate interior decoration proposals that meet the user's requests and conditions.

[0877] The system for implementing this invention starts by acquiring images of a room using the user's smartphone or camera and sending the image data to a server. The server uses image recognition technology to specify the room category from the acquired images and plans the interior decoration based on the specified category. The planned interior decoration is generated as visual information and presented to the user.

[0878] The server analyzes the user's facial expressions and voice tone using the smartphone's camera and microphone to recognize the user's emotions. Image recognition libraries (e.g., OpenCV) and emotion recognition APIs (e.g., Microsoft Azure Emotion API) are used for emotion recognition. Based on the recognized emotions, the server suggests interior decorations that correspond to those emotions.

[0879] Furthermore, based on the budget information provided by the user, the server selects interior decorations that can be purchased within the budget and provides optimal suggestions. A generative AI model (e.g., OpenAI GPT-3) is used to generate these suggestions. The generated suggestions are provided along with a link to an online platform for easy access by the user.

[0880] As a concrete example, consider a scenario where a user takes a photo of their living room with their smartphone and enters a budget of 50,000 yen. The server detects that the user has a cheerful expression and suggests bright and colorful interior decorations. The suggested items are displayed with links to online platforms.

[0881] An example of a prompt message is: "If the user's emotion is joy, please suggest bright and colorful interior items that can be purchased within a budget of 50,000 yen."

[0882] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[0883] Step 1:

[0884] The user takes a picture of the room using their smartphone camera. The captured image is sent from the device to the server. The input is the image data of the room, and the output is the image data sent to the server.

[0885] Step 2:

[0886] The server analyzes the received image data using image recognition technology (e.g., OpenCV) to specify the room category. The input is image data of the room, and the output is the specified category information. As a data processing step, features are extracted from the image and classified into categories.

[0887] Step 3:

[0888] The server plans the interior decoration based on the specified category. The planning process includes selecting appropriate furniture and decorations from a database. The input is category information, and the output is a list of the planned interior decorations.

[0889] Step 4:

[0890] The server generates visual information of the planned interior decoration. This visual information is represented as images or 3D models. The input is a list of interior decorations, and the output is the visual information. As a data operation, the selected items are visually arranged.

[0891] Step 5:

[0892] The user's emotions are recognized using their smartphone's camera and microphone. The server uses an emotion recognition API (e.g., Microsoft Azure Emotion API) to analyze the user's facial expressions and voice tone. The input is the user's facial expression data and voice data, and the output is the recognized emotion information.

[0893] Step 6:

[0894] The server suggests interior decorations that correspond to the recognized emotions. The input is emotional information, and the output is a suggestion for interior decorations that match those emotions. As a data processing step, a style appropriate to the emotion is selected.

[0895] Step 7:

[0896] The user enters budget information. The server selects interior decorations that can be purchased within the user's budget and provides optimal suggestions. The input is budget information, and the output is a suggestion of interior decorations that fit the budget. As part of the data calculation, items are selected considering price information.

[0897] Step 8:

[0898] The server generates the final proposal using a generative AI model (e.g., OpenAI GPT-3). The input consists of category information, sentiment information, and budget information, and the output is a final interior design proposal. Prompts are used to guide the AI ​​model in generating the proposal.

[0899] Step 9:

[0900] The server presents the generated proposals to the user and provides a link to the online platform. The input is the final interior design proposal, and the output is the display of the proposals to the user and a purchase link. Specifically, the server sends the proposal content to the user's device.

[0901] (Other examples)

[0902] Since this is the same as the specific processing described in the other embodiments of the first embodiment above, the explanation will be omitted.

[0903] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0904] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0905] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[0906] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0907] [Third Embodiment]

[0908] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0909] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0910] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0911] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0912] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0913] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0914] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0915] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0916] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0917] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0918] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0919] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0920] "Example of form 1"

[0921] One embodiment of the present invention provides a system that includes a camera module for taking pictures of a room, an interior planning module for specifying a genre based on the taken pictures and planning an interior that matches the specified genre, and an image generation module for generating image photos of the planned interior.

[0922] "Example of form 2"

[0923] As another embodiment of the present invention, a system is provided that further includes a site guidance module that guides users to a site where the generated interior image can be purchased. Specifically, it provides a link to an online shopping site where the interior item can be purchased, based on information about the interior item included in the generated image.

[0924] "Example of form 3"

[0925] In yet another embodiment of the present invention, a system is provided that further includes a coordination suggestion module that proposes an interior design that fits the budget provided by the user. Specifically, based on the budget provided by the user, the system selects interior items that can be purchased within that budget and proposes a coordination that combines them.

[0926] The following describes the processing flow for each example form.

[0927] "Example of form 1"

[0928] Step 1: The user takes a picture of the room. This is done using a camera module built into the system.

[0929] Step 2: Based on the photos taken, the user specifies the interior design genre. The genre is specified on the system's user interface.

[0930] Step 3: Plan the interior design to match the specified genre. This planning is done automatically by the system's interior planning module.

[0931] Step 4: Generate image photos of the planned interior. This generation is performed automatically by the system's image generation module.

[0932] "Example of form 2"

[0933] Step 1: Based on the generated interior image photos, the system redirects users to websites where the corresponding interior items can be purchased. This redirection is performed automatically by the system's site redirection module.

[0934] Step 2: Specifically, based on the information about the interior items included in the generated image, provide links to online shopping sites where those items can be purchased. (Example 3)

[0935] Step 1: Based on the budget provided by the user, the system proposes an interior design coordinated to fit that budget. This proposal is automatically generated by the system's coordination proposal module.

[0936] Step 2: Specifically, based on the budget provided by the user, select interior items that can be purchased within that budget and propose a coordinated look using those items.

[0937] (Example 1)

[0938] Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0939] In modern living spaces, efficiently proposing interior designs that meet individual preferences and budgets is difficult. Traditional methods often require expert advice and are time-consuming and costly. Furthermore, visualizing interior designs is challenging, making it difficult for clients to form concrete images.

[0940] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0941] In this invention, the server includes means for acquiring an image of a space using a camera; means for analyzing the acquired image and determining the category of the space based on the features in the image; means for planning an interior design that fits the determined category; means for generating a visual representation of the planned interior design using a generative AI model; and means for transmitting the generated visual representation to the user's device. This allows the user to quickly and efficiently visualize interior designs that suit their preferences and budget, and to have a concrete image of them.

[0942] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and smartphones.

[0943] A "spatial image" is digital image data that contains visual information about a specific place or room.

[0944] "Image analysis" is a technique that processes acquired image data to identify features and elements within the image.

[0945] A "spatial category" is a classification that indicates the style or theme of a space, determined based on image analysis.

[0946] "Interior design" is the process of planning the placement of furniture and decorative items to suit a particular space.

[0947] A "generative AI model" is an algorithm that uses artificial intelligence technology to generate new visual representations from data.

[0948] "Visual representation" refers to images or illustrations that visually show the results of the interior design.

[0949] "User's device" refers to a device used to receive and display the generated visual representation, and includes smartphones, computers, and other similar devices.

[0950] An "online platform" is a website or application that provides goods and services via the internet.

[0951] "Budget information" refers to data about the amount of money users can spend on interior design.

[0952] This invention is a system that allows a user to take a photograph of a room and then proposes an interior design based on that photograph. The user uses a terminal's camera to acquire an image of the room. The acquired image is sent from the terminal to a server. The server uses image analysis software to analyze the features in the image and determine the category of the space. OpenCV, a Python image processing library, can be used for this analysis.

[0953] Next, the server plans the interior design based on the determined category. This involves referencing templates in the database and selecting a design that fits the category. The server then uses a generative AI model to generate a visual representation of the planned interior design. Generative AI models such as DALL-E and Stable Diffusion can be used for this generation.

[0954] The generated visual representation is sent from the server to the user's terminal. The user can view the visual representation on their terminal and use it as a reference for interior design. For example, if the user takes a picture of their living room, the server will determine the category to be "modern" and suggest a modern interior design. The generation AI model can generate an appropriate visual representation by inputting a prompt such as, "Generate a modern living room interior design."

[0955] This system allows users to quickly and efficiently visualize interior designs that suit their preferences and budget, giving them a concrete image of what they want.

[0956] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0957] Step 1:

[0958] The user takes an image of the room using the device's camera. The user opens the smartphone's camera app and takes a picture so that the entire room is visible. The captured image is sent from the device to the server. The input is the photo of the room, and the output is the image data sent to the server.

[0959] Step 2:

[0960] The server analyzes the received image data. Using image analysis software, it identifies features within the image and determines the spatial category. OpenCV, a Python image processing library, can be used for this analysis. The input is the image data sent to the server, and the output is the determined spatial category.

[0961] Step 3:

[0962] The server plans the interior design based on the determined category. It refers to templates in the database and selects a design that fits the category. The input is the category of the space, and the output is the planned interior design.

[0963] Step 4:

[0964] The server uses a generative AI model to generate a visual representation of the planned interior design. DALL-E and Stable Diffusion can be used as the generative AI model. The prompt "Generate a modern living room interior design" is entered, and the visual representation is generated. The input is the planned interior design and the prompt, and the output is the generated visual representation.

[0965] Step 5:

[0966] The server sends the generated visual representation to the user's terminal. The user can then view the visual representation on their terminal and use it as a reference for interior design. The input is the generated visual representation, and the output is the visual representation displayed on the user's terminal.

[0967] (Application Example 1)

[0968] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0969] Modern consumers face the challenge of finding the perfect interior design for their living space from a vast array of options. Furthermore, online purchases without in-store inspection can lead to discrepancies between the image and the actual product. Additionally, receiving optimal interior design suggestions that fit within one's budget can be difficult.

[0970] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0971] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations that match the specified category, and means for generating visual information of the planned interior decorations. This allows consumers to visually confirm interiors that suit their own rooms, actually check them in stores, and receive optimal suggestions that fit their budget.

[0972] "Means for acquiring images of a room" refers to a device or method for a user to photograph their living space and import that image data into the system.

[0973] "Means for specifying a category and planning interior decoration to match the specified category" refers to an apparatus or method for analyzing acquired image data, selecting an appropriate interior style based on the user's preferences and the characteristics of the room, and creating a decoration plan that matches that style.

[0974] "Means for generating visual information of planned interior decoration" refers to an apparatus or method for generating an image of the interior in a form that can be visually confirmed by the user, based on the selected interior style.

[0975] "Means for proposing goods" refers to a device or method for proposing specific products or decorative items to a user based on a generated interior image.

[0976] "Means of guiding users to a place where they can actually see the product" refers to a device or method for guiding users to a store or exhibition space so that they can actually see the proposed product or decorative item.

[0977] "Means of providing purchasable information" refers to a device or method that provides users with information to purchase proposed goods or decorative items.

[0978] "A means of proposing interior decoration that matches the presented funds based on the presented funds" refers to a device or method for proposing the optimal interior plan within the budget presented by the user.

[0979] A description of the embodiment for carrying out the invention will be given.

[0980] The system that realizes this invention mainly consists of a user terminal and a server. The user terminal is a mobile information terminal such as a smartphone or tablet, and is equipped with a camera module for acquiring images of the room. The user uses this terminal to take pictures of their room.

[0981] The captured image data is sent from the terminal to the server. The server uses image recognition software (e.g., TensorFlow) to analyze the image data and extract the characteristics of the room. Based on this analysis, the server uses a generative AI model (e.g., OpenAI's DALL-E) to select an interior style that suits the user's preferences and the characteristics of the room, and generates visual information of the interior decoration that matches that style.

[0982] The generated visual information is sent to the user's device, where the user can visually confirm it. Furthermore, the server suggests specific products and decorative items to the user based on the generated interior image. These suggestions also include information directing the user to stores or exhibition spaces where they can actually see the products.

[0983] For example, if a user requests a "modern living room," the AI ​​model will receive a prompt such as, "Generate interior design suggestions for a modern living room. The room photos are below." This allows the user to visually see interior designs that suit their room and then actually check them out in a store. The system will also propose an optimal interior design plan based on the user's stated budget.

[0984] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0985] Step 1:

[0986] The user takes a picture of the room using the device's camera module. The input is the image data of the room taken by the user. The output is the captured image data being saved to the device.

[0987] Step 2:

[0988] The device sends the captured image data to the server. The input is the image data stored on the device. The output is the image data transferred to the server.

[0989] Step 3:

[0990] The server analyzes the received image data using image recognition software (e.g., TensorFlow). The input is the image data transferred to the server. Feature extraction is performed as part of the data processing. The output is data indicating the characteristics of the room.

[0991] Step 4:

[0992] The server uses a generative AI model (e.g., OpenAI's DALL-E) to select an interior style that matches the user's preferences and the characteristics of the room, based on room feature data. The input is room feature data. The data calculation involves selecting an interior style. The output is data of the selected interior style.

[0993] Step 5:

[0994] The server generates visual information of the interior decoration based on the selected interior style. The input is data of the selected interior style. As a data processing step, visual information is generated. The output is the generated visual information.

[0995] Step 6:

[0996] The server sends the generated visual information to the user's terminal. The input is the generated visual information. The output is the visual information transferred to the user's terminal.

[0997] Step 7:

[0998] The user reviews the visual information received on the device and visually evaluates the proposed interior. The input is the visual information displayed on the device. The output is the user's evaluation.

[0999] Step 8:

[1000] The server suggests specific products and decorations to the user based on the generated interior image and provides information guiding them to stores or exhibition spaces where they can actually view the products. The input is the generated interior image. The data calculation involves generating product suggestions and guidance information. The output is the suggested product information and guidance information.

[1001] (Example 2)

[1002] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1003] Traditional interior design systems have faced challenges such as difficulty in receiving specific interior design proposals based on the user's desired style and theme, and a lack of information for actually purchasing the suggested interior items. Furthermore, providing optimal interior coordination within the user's budget has also been difficult.

[1004] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1005] In this invention, the server includes means for acquiring images, means for specifying categories based on the acquired images and planning spatial decorations suitable for the specified categories, means for generating visual information of the planned spatial decorations, means for analyzing items included in the generated visual information, and means for providing a connection to an information source from which the analyzed items can be obtained. This allows the user to receive interior design suggestions based on their desired style and to easily purchase the suggested items. It is also possible to provide optimal interior coordination according to the user's budget.

[1006] "Means for acquiring images" refers to a device or method for collecting visual information provided by a user in digital format.

[1007] "Means for specifying categories" refers to an apparatus or method for determining a classification related to a specific style or theme based on acquired images.

[1008] "Means for planning spatial decoration" refers to an apparatus or method for constructing an interior design suitable for a specified category.

[1009] "Means for generating visual information" refers to an apparatus or method for creating a planned spatial decoration as a digital image.

[1010] "Means for analyzing articles" refers to a device or method for identifying interior items contained in generated visual information and extracting their characteristics.

[1011] "Means of providing access to information sources" refers to a device or method for generating and providing to a user a link to an online platform where the analyzed items can be purchased.

[1012] The following systems are conceivable as embodiments for carrying out this invention.

[1013] The user uses a terminal to enter prompts to generate interior design images. These prompts can specify the style or theme the user desires. For example, they might enter prompts such as "modern living room" or "Nordic-style bedroom."

[1014] The terminal sends the prompt message received from the user to the server. The server uses a generative AI model to generate an image of the interior based on the prompt message. This generative AI model can use image generation technologies such as DALL-E or Midjourney.

[1015] The generated images are analyzed by a server. The server uses image recognition technology to identify interior items contained in the images and extract their characteristics. This analysis reveals what the identified items are.

[1016] Next, the server provides a connection to information sources where the identified interior items can be purchased, based on the analysis results. Specifically, it generates and provides links to online shopping platforms. Database search technology can be used to generate these links. For example, it could provide links to platforms such as Amazon or IKEA.

[1017] Users can view images and purchase links generated through their devices. By clicking on the link for an item they are interested in, users can easily access the purchase page and actually buy the item.

[1018] This system allows users to obtain ideas for their desired interior design while simultaneously facilitating the process of actually purchasing those items.

[1019] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1020] Step 1:

[1021] The user uses a terminal to input prompt text for interior design. This prompt text specifically expresses the user's desired style or theme. For example, it might be something like "vintage-style dining room." This prompt text serves as the basic data for subsequent processing.

[1022] Step 2:

[1023] The terminal sends the prompt message received from the user to the server. The server receives this prompt message as input and requests image generation from the generative AI model. The generative AI model generates an image of the interior based on the prompt message. In this process, the content of the prompt message is analyzed and data processing is performed to generate an appropriate image. The generated image is obtained as output.

[1024] Step 3:

[1025] The server analyzes the generated image. It receives an image as input and uses image recognition technology to identify the interior items contained in the photograph. This analysis extracts the type and characteristics of the items. The output provides information about the identified items. Specifically, items such as sofas and tables are identified from the photograph.

[1026] Step 4:

[1027] The server provides a connection to information sources where the identified interior items can be purchased, based on the analysis results. It receives item information as input and uses database search technology to generate links to online shopping platforms. The output is a purchase link. Specifically, links to platforms like Amazon and IKEA are generated.

[1028] Step 5:

[1029] The terminal displays the generated image and purchase link received from the server to the user. The user can view the displayed image and click the link for an item of interest to access the purchase page. It receives the generated image and purchase link as input and provides visual information to the user as output. Specifically, the user can click the link and proceed with the purchase process.

[1030] (Application Example 2)

[1031] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1032] Modern consumers want to easily find and purchase interior decorations that suit their living spaces. However, in reality, finding the right items often requires a lot of time and effort. Furthermore, the need to compare multiple online platforms to find available items makes efficient purchasing difficult.

[1033] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1034] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations suitable for the specified category, means for generating visual information of the planned interior decorations, and means for identifying items included in the generated visual information and providing connection information to an online platform where the items can be acquired. This makes it possible for users to easily find and efficiently purchase interior decorations that suit their rooms.

[1035] "Means for acquiring images of a room" refers to a function that allows users to take pictures of a room using a mobile device and import that image data into the system.

[1036] "A means of specifying a category and planning interior decoration suitable for that category" refers to a function that selects an interior decoration style or theme based on acquired images and creates a decoration plan that matches it.

[1037] "Means for generating visual information for interior decoration" refers to a function that generates images and graphics to visually represent the planned interior decoration.

[1038] "Means for identifying items and providing connection information to online platforms where those items can be obtained" refers to a function that identifies individual items included in the generated visual information and provides users with links and information to online platforms where those items can be purchased.

[1039] The system for implementing this invention begins with a user acquiring an image of a room using a mobile device and sending that image to a server. The server uses image recognition software (e.g., TensorFlow, OpenCV) to analyze the objects in the image and specify their categories. Based on the specified categories, it uses a generative AI model (e.g., OpenAI's GPT model) to create a suitable interior decoration plan.

[1040] Next, the server generates visual information to visually represent the planned interior decoration. This visual information includes interior items that are suitable for the user's room. The server identifies each item included in the generated visual information and provides connection information to the online platform where they can be obtained. This allows the user to easily purchase items they are interested in.

[1041] As a concrete example, when a user takes a picture of their living room and sends it to the system, the server recognizes items such as a sofa, table, and lamp. For each item, it generates links to online platforms such as Amazon and Rakuten Market and provides them to the user. By clicking on the provided links, the user can directly access the purchase page for the corresponding item.

[1042] An example of a prompt would be, "Please tell me where I can buy the same sofa shown in this picture." By inputting this prompt into the AI ​​model, a link to the appropriate online platform will be generated.

[1043] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1044] Step 1:

[1045] The user takes a picture of the room using a mobile device. The captured image is saved on the device.

[1046] Step 2:

[1047] The terminal sends the captured image to the server. The input is image data of the room, and the output is the transfer of the image data to the server.

[1048] Step 3:

[1049] The server analyzes the received image data using image recognition software (e.g., TensorFlow, OpenCV). The input is image data, and the output is category information of the objects in the image. The server identifies the objects in the image and assigns each category to them.

[1050] Step 4:

[1051] The server uses a generative AI model to create interior decoration plans based on specified categories. The input is category information for items, and the output is an interior decoration plan. The server generates decoration ideas suitable for each category.

[1052] Step 5:

[1053] The server generates visual information to visually represent the planned interior decoration. The input is the interior decoration plan, and the output is visual information (image data). The server generates images to visualize the decoration plan.

[1054] Step 6:

[1055] The server identifies each item included in the generated visual information and provides connection information to the online platform where they can be obtained. The input is visual information, and the output is link information to the online platform. The server generates a purchase link for each item.

[1056] Step 7:

[1057] Users receive link information provided through their device and select items they are interested in. By clicking on the link, users can access the purchase page for the selected item.

[1058] (Example 3)

[1059] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1060] Traditional interior design systems struggled to provide optimal suggestions tailored to the user's budget, and verifying whether the proposed designs were actually affordable was cumbersome. Furthermore, there was a lack of effective means to utilize user feedback to improve the system.

[1061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[1062] In this invention, the server includes means for acquiring images, means for specifying categories based on the acquired images and planning decorative items suitable for the specified categories, means for generating visual information of the planned decorative items, means for receiving budget information from the user, selecting purchasable decorative items based on the budget information and generating suggestions combining the selected decorative items, and means for suggesting the optimal combination of the selected decorative items using a generation AI model. This makes it possible to suggest the optimal interior coordination according to the user's budget, and also makes it easier to confirm that the suggested coordination is actually purchasable. Furthermore, user feedback can be received and used to improve the system.

[1063] "Means for acquiring images" refers to a device or method for visually recording the user's environment or the state of a room.

[1064] "Means of specifying categories" refers to methods for classifying interior styles and themes based on acquired images.

[1065] "Means of planning decorative items" refers to methods for selecting interior items suitable for a specified category and considering their placement.

[1066] "Means of generating visual information" refers to methods for visually representing the planned layout and design of an interior.

[1067] "Means of receiving budget information" refers to methods for obtaining information about the amount of money that users are willing to spend.

[1068] "Methods for selecting decorative items" refers to methods for selecting interior items that are affordable within the user's budget.

[1069] "Means for generating proposals" refers to a method for combining selected decorative items to propose the optimal interior design for the user.

[1070] "Methods using generative AI models" refer to methods that utilize artificial intelligence technology to calculate and propose the optimal combination of selected decorative items.

[1071] "Means of receiving feedback" refers to methods for obtaining opinions and evaluations from users and using them to improve the system.

[1072] A description of embodiments for carrying out this invention will be given.

[1073] The user first accesses the interface using a terminal and enters budget information. The terminal sends this input to the server. The server then searches its database based on the budget information received from the user. Specifically, it uses a database management system such as MySQL to query for interior items that can be purchased within the budget.

[1074] The server generates a coordinated look by combining interior items selected from the search results. This process can utilize a generative AI model. Specifically, the AI ​​model is given a prompt message such as, "Please suggest the best coordinated look using the selected items within the budget," and the AI ​​generates the suggestions.

[1075] The generated outfit suggestions are sent from the server to the terminal. The terminal displays the suggestions to the user. The user can review the suggested outfits and provide feedback as needed. The terminal sends this feedback to the server to help improve the system.

[1076] As a concrete example, if a user specifies a budget of "50,000 yen," the system will select items such as a sofa, table, and lamp that can be purchased within that budget and propose a coordinated look combining them. In this way, the user can receive interior design suggestions that match their budget. The specific processing flow in Example 3 will be explained using Figure 15.

[1077] Step 1:

[1078] The user accesses the interface using their device and enters budget information. Specifically, the user enters the budget amount into a web form or app input field and clicks the "Submit" button. The entered budget information is sent from the device to the server.

[1079] Step 2:

[1080] The server searches the database based on the budget information received from the user. Specifically, it uses a database management system such as MySQL to query for interior items that can be purchased within the budget. The input is the budget information, and the output is a list of items that can be purchased within the budget. The server considers the price information of the items and selects only those items that do not exceed the budget.

[1081] Step 3:

[1082] The server generates a coordinated interior design by combining selected interior items from the search results. A generative AI model can be used for this process. Specifically, the AI ​​model is given the prompt, "Please suggest the best coordinated design using items selected within the budget," and the AI ​​generates the suggestion. The input is a list of items, and the output is the coordinated design suggestion.

[1083] Step 4:

[1084] The server sends the generated outfit suggestions to the terminal. The terminal displays the suggestions to the user. Specifically, the terminal displays images and descriptions of the outfits on a web page or app screen, allowing the user to review them. The input is the outfit suggestions, and the output is the visual display to the user.

[1085] Step 5:

[1086] Users can review suggested outfits and provide feedback. The device sends this feedback to the server to help improve the system. Specifically, users can choose options such as "I like this outfit" or "I want to see other suggestions." The input is user feedback, and the output is information related to system improvements.

[1087] (Application Example 3)

[1088] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1089] Modern consumers face the challenge of achieving optimal interior design within a limited budget. Furthermore, selecting and combining interior items requires specialized knowledge, making it a high hurdle for the average consumer. Therefore, there is a need for a system that can easily propose optimal interior design solutions within a given budget.

[1090] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1091] In this invention, the server includes means for acquiring images of a space using a camera; means for selecting a category based on the acquired images and planning decorative items that fit the selected category; and means for receiving budget information from the user, selecting decorative items that can be purchased within that budget, and proposing the optimal combination. This makes it possible for consumers to easily achieve the optimal interior design within their budget.

[1092] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and smartphones.

[1093] "Spatial images" are photographs or image data that contain visual information about a room or a specific area.

[1094] A "category" is a standard used to classify interior items, and is selected based on style and purpose.

[1095] "Decorative items" refer to furniture, accessories, and other items used in interior design.

[1096] "Planning" is the process of creating an interior design by combining decorative items that fit the selected category.

[1097] A "visualized image" is an image that visually represents the planned arrangement and combination of decorative items.

[1098] "Budget information" refers to data about the amount of money users can spend on interior design.

[1099] An "online platform" refers to a website or application that allows users to purchase goods over the internet.

[1100] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and select decorative items that can be purchased within a budget.

[1101] The system for implementing this invention mainly consists of a server and a user terminal. The server has the function of acquiring images of a space using a camera and receives image data transmitted from the user terminal. Based on the acquired images, the server selects a category and processes items to plan decorative items that fit the selected category. In this process, the server uses a generative AI model to consider budget information provided by the user and selects decorative items that can be purchased within the budget.

[1102] The server generates a visualization of the planned decorations and sends it to the user's terminal. The user can view the visualization on their terminal and purchase the suggested decorations through the online platform. The server generates and provides a link to the online platform to the user.

[1103] As a concrete example, if a user takes a picture of their living room with their smartphone and enters a budget of 50,000 yen, the server analyzes the image and selects a category suitable for the living room. The generating AI model then selects decorative items such as sofas, tables, and curtains that can be purchased within 50,000 yen and suggests the optimal combination. The user can then review the suggested coordination and purchase the items they like.

[1104] An example of a prompt for a generating AI model is, "Please select interior items that can be purchased for a budget of 50,000 yen and suggest a living room coordination." This prompt allows the AI ​​model to provide optimal suggestions tailored to the user's needs.

[1105] The flow of the specific processing in Application Example 3 will be explained using Figure 16.

[1106] Step 1:

[1107] The user takes a picture of the room using their smartphone and sends it to the server through the application. The input is the image data of the room, and the output is the transfer of the image data to the server. In this step, the smartphone's camera function is used to acquire the image and upload it to the server via the internet.

[1108] Step 2:

[1109] The server analyzes the received image data and uses image recognition technology to select a room category. The input is image data sent by the user, and the output is the selected category information. The server uses an image analysis algorithm to extract room features and determine the appropriate category.

[1110] Step 3:

[1111] The user enters budget information through the application and sends it to the server. The input is the budget information specified by the user, and the output is the transfer of budget data to the server. In this step, the user specifies the budget using the application's input form and sends it to the server.

[1112] Step 4:

[1113] The server uses a generative AI model to select purchasable decorative items based on the chosen category and budget information. The input is category information and budget information, and the output is a list of selected decorative items. The server inputs prompts into the AI ​​model to select the most suitable decorative item within the budget.

[1114] Step 5:

[1115] The server generates a visualization image based on the selected ornaments and sends it to the user terminal. The input is a list of ornaments, and the output is a visualization image. The server uses a graphics generation algorithm to visualize the arrangement of the ornaments and generate the image.

[1116] Step 6:

[1117] The user views visualized images on their device and selects their preferred decorative items. The input is the visualized images, and the output is the user's selection information. The user uses the application interface to review the suggested decorative items and select the items they wish to purchase.

[1118] Step 7:

[1119] The server generates and provides to the user a link to the online platform corresponding to the selected decorative item. The input is the user's selection information, and the output is a link to the online platform. The server generates a link to the purchase page for the selected item and sends it to the user.

[1120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1121] "Example of form 1"

[1122] One embodiment of the present invention is a system that combines a user emotion engine with a system that recognizes the user's emotions. This system takes a photo of a room and specifies a genre based on the photo. It plans an interior that matches the specified genre and generates an image of the planned interior. Furthermore, based on the generated image of the interior, it guides the user to a website where the corresponding interior can be purchased. Finally, based on the budget provided by the user, it proposes an interior coordination that fits the budget. In addition to this series of steps, a user emotion engine is incorporated to recognize the user's emotions. Specifically, it estimates the user's emotions from their facial expressions, tone of voice, and speed of operation when they operate the system, and proposes an interior that matches that emotion. For example, if the user is showing feelings of joy, it will propose an interior with a bright and cheerful atmosphere. Conversely, if the user is feeling down, it will propose a calm and soothing interior. This makes it possible to provide more personalized interior proposals that respond to the user's emotions.

[1123] "Example of form 2"

[1124] One embodiment of the present invention is a system that combines a user emotion engine with a system that recognizes the user's emotions. This system takes a photo of a room and specifies a genre based on the photo. It plans an interior that matches the specified genre and generates an image of the planned interior. Furthermore, based on the generated image of the interior, it guides the user to a website where the corresponding interior can be purchased. Finally, based on the budget provided by the user, it proposes an interior coordination that fits the budget. In addition to this series of steps, a user emotion engine is incorporated to recognize the user's emotions. Specifically, it estimates the user's emotions from their facial expressions, tone of voice, and speed of operation when they operate the system, and proposes an interior that matches that emotion. For example, if the user is showing feelings of joy, it will propose an interior with a bright and cheerful atmosphere. Conversely, if the user is feeling down, it will propose a calm and soothing interior. This makes it possible to provide more personalized interior proposals that respond to the user's emotions.

[1125] "Example of form 3"

[1126] One embodiment of the present invention is a system that combines a user emotion engine with a system that recognizes the user's emotions. This system takes a photo of a room and specifies a genre based on the photo. It plans an interior that matches the specified genre and generates an image of the planned interior. Furthermore, based on the generated image of the interior, it guides the user to a website where the corresponding interior can be purchased. Finally, based on the budget provided by the user, it proposes an interior coordination that fits the budget. In addition to this series of steps, a user emotion engine is incorporated to recognize the user's emotions. Specifically, it estimates the user's emotions from their facial expressions, tone of voice, and speed of operation when they operate the system, and proposes an interior that matches that emotion. For example, if the user is showing feelings of joy, it will propose an interior with a bright and cheerful atmosphere. Conversely, if the user is feeling down, it will propose a calm and soothing interior. This makes it possible to provide more personalized interior proposals that respond to the user's emotions.

[1127] The following describes the processing flow for each example form.

[1128] "Example of form 1"

[1129] Step 1: The user takes a photo of the room.

[1130] Step 2: The system specifies a genre based on the photos it has taken.

[1131] Step 3: The system plans an interior design that matches the specified genre and generates image photos of the planned interior.

[1132] Step 4: Based on the interior image photos generated by the system, users are directed to a website where the corresponding interior items can be purchased.

[1133] Step 5: Based on the budget provided by the user, the system proposes interior design ideas that fit within that budget.

[1134] Step 6: The system uses an emotion engine to recognize the user's emotions, estimating their feelings from their facial expressions, tone of voice, and speed of operation while they interact with the system.

[1135] Step 7: The system suggests interior design elements that match the user's estimated emotions. For example, if the user is expressing joy, the system suggests a bright and cheerful interior design. Conversely, if the user is feeling down, the system suggests a calming and soothing interior design.

[1136] (Example 1)

[1137] Next, we will describe Embodiment 1 of Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1138] In modern living spaces, there is a demand for personalized interior design proposals that cater to the individual user's emotions and budget. However, conventional systems struggle to provide interior design proposals that take user emotions into account, and they also have difficulty quickly providing optimal proposals that fit within the budget.

[1139] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1140] In this invention, the server includes means for acquiring images of a space using a camera; means for identifying categories based on the acquired images and planning interiors that fit the identified categories; means for generating visual information of the planned interiors; and means for recognizing the user's emotions using an emotion analysis device and adjusting interior design suggestions based on the recognized emotions. This enables personalized interior design suggestions that are tailored to the user's emotions and budget.

[1141] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and sensors.

[1142] A "spatial image" is digital image data that includes visual information of a room or a specific location.

[1143] "Category" refers to the genre or style of interior design classified based on the characteristics of the space.

[1144] "Interior design" refers to all aspects of design, including the decoration of rooms and spaces, and the arrangement of furniture.

[1145] "Planning methods" refer to the processes and methods for designing and proposing interiors that fit a specific category.

[1146] "Visual information" refers to images and illustrations that visually represent the planned interior design.

[1147] An "emotion analysis device" is a device designed to recognize the user's emotions and has the function of analyzing facial expressions, tone of voice, operation speed, etc.

[1148] "User emotions" refers to the psychological state or feelings that users exhibit when operating the system.

[1149] "Means of adjusting proposals" refers to methods or processes for modifying or optimizing interior design proposals based on perceived emotions.

[1150] The following system is constructed as an embodiment of this invention.

[1151] The server first receives an image of the space transmitted from the terminal. This image is taken by the user using a camera. The server uses a generative AI model to analyze the received image and extract spatial features from it. Based on these features, the server identifies a category and plans an interior design that fits that category.

[1152] The interior planning module is used for interior design planning. This module searches the database for relevant interior items and generates a list of furniture and decorations that fit the categories. The server then uses the image generation module to generate visual information of the planned interior. This visual information is intended to provide the user with a concrete image of the interior.

[1153] Furthermore, the server uses an emotion analysis device to recognize the user's emotions. It analyzes data such as the user's facial expressions, tone of voice, and operation speed to estimate the user's emotions. Based on this emotional information, the server adjusts its interior design suggestions. For example, if the user is showing an emotion of joy, the server sends a prompt message to the AI ​​model saying, "Please suggest an interior with a bright and cheerful atmosphere," and then makes a suggestion.

[1154] As a concrete example, here is an example of a prompt message to be input into a generative AI model: "Analyze the room photos taken by the user and specify a genre. Plan an interior design that matches that genre and generate an image photo. Also, recognize the user's emotions and provide interior design suggestions that match those emotions."

[1155] In this way, the server can provide personalized interior design suggestions tailored to the user's emotions and budget.

[1156] The flow of the specific processing in Example 1 will be explained using Figure 17.

[1157] Step 1:

[1158] The user takes a picture of the room using the camera on their device. The captured image is sent from the device to the server. The input is the image data of the room, and the output is the transmission of the image data to the server.

[1159] Step 2:

[1160] The server analyzes the received image data. Using a generative AI model, it extracts spatial features from the image and identifies categories. The input is the received image data, and the output is the identified category information.

[1161] Step 3:

[1162] The server uses an interior planning module to plan interiors that fit a specified category. It searches the database for relevant interior items and generates a list of furniture and decorations that match the category. The input is category information, and the output is a list of interior items.

[1163] Step 4:

[1164] The server uses an image generation module to generate visual information of the planned interior. It sends a prompt message to the generation AI model saying, "Create an image based on this interior list," and creates a realistic image. The input is a list of interior items, and the output is visual information of the interior.

[1165] Step 5:

[1166] The server recognizes the user's emotions using an emotion analysis device. It analyzes data such as the user's facial expressions, voice tone, and operation speed to estimate their emotions. The input is the user's operation data, and the output is the estimated emotion information.

[1167] Step 6:

[1168] The server adjusts interior design suggestions based on the recognized emotional information. For example, if the user indicates a feeling of joy, the server sends a prompt message to the AI ​​model saying, "Please suggest an interior with a bright and cheerful atmosphere," and then makes a suggestion. The input is emotional information, and the output is an adjusted interior design suggestion.

[1169] Step 7:

[1170] The server uses the generated visual information of the interior design to guide users to information sources where that interior design is available for purchase. Users can view this information through their terminal and purchase items that interest them. The input is the visual information of the interior design, and the output is a guide to purchase information.

[1171] (Application Example 1)

[1172] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1173] Modern consumers are seeking personalized interior design proposals that cater to their individual emotions and budgets. However, traditional interior design proposal systems have faced challenges in considering user emotions and providing optimal proposals within a given budget.

[1174] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1175] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations that match the specified category, means for generating visual information of the planned interior decorations, and means for recognizing the user's emotions and suggesting interior decorations that correspond to those emotions. This makes it possible to suggest personalized interior decorations that correspond to the user's emotions and budget.

[1176] "Means for acquiring images of a room" refers to a device or method for photographing the interior of a room and acquiring the resulting image data.

[1177] "Means for specifying categories" refers to a device or method for classifying the style or theme of interior decoration based on acquired image data.

[1178] "Means for planning interior decoration" refers to an apparatus or method for devising the appropriate arrangement of furniture and decorative items based on a specified category.

[1179] "Means for generating visual information" refers to a device or method for visually representing the image of a planned interior decoration.

[1180] "Means for recognizing user emotions" refers to a device or method for estimating a user's emotions from their facial expressions, tone of voice, etc.

[1181] "Means for proposing interior decoration" refers to a device or method for presenting optimal interior decoration in accordance with the recognized emotions of the user.

[1182] The system for implementing this invention begins with the user acquiring images of a room using a smartphone or smart glasses. The device uses a camera module to capture images of the room and sends the data to a server. The server analyzes the acquired images using image recognition technology and specifies the category of interior decoration. Image recognition software such as the Google Cloud Vision API can be used for this purpose.

[1183] Next, the server plans the interior decoration based on the specified category. Visual information of the planned interior decoration is generated using a generative AI model. By utilizing generative AI models such as OpenAI GPT-3, it is possible to provide users with visually appealing images of interior decoration.

[1184] Furthermore, the device analyzes the user's facial expressions and tone of voice to recognize their emotions. Using technologies such as Microsoft Azure Face API and IBM Watson Speech to Text, it can estimate the user's emotions. The server then suggests optimal interior decorations based on the recognized emotions.

[1185] For example, if a user takes a picture of their living room using smart glasses, the server analyzes the image and assigns it the "modern style" category. If the server detects that the user is smiling, it suggests furniture and decorations in bright colors. An example of a prompt might be, "Suggest products that match the interior of this photo. The user's emotion is joy."

[1186] The flow of a specific process in Application Example 1 will be explained using Figure 18.

[1187] Step 1:

[1188] The user takes pictures of the room using a smartphone or smart glasses. The input is image data acquired through the camera module, and the output is that same image data. The image is captured when the user presses the capture button.

[1189] Step 2:

[1190] The device sends the captured image data to the server. The input is the image data acquired in step 1, and the output is the image data sent to the server. The device uploads the data over the network.

[1191] Step 3:

[1192] The server analyzes the received image data using image recognition technology and specifies the category of interior decoration. The input is the image data sent to the server, and the output is the specified category. The server uses the Google Cloud Vision API to analyze the image and determine the category.

[1193] Step 4:

[1194] The server plans interior decorations based on specified categories and generates visual information using a generative AI model. The input is the specified categories, and the output is the generated visual information. The server uses OpenAI GPT-3 to generate images of visually appealing interior decorations.

[1195] Step 5:

[1196] The device analyzes the user's facial expressions and voice tone to recognize emotions. Input is the user's facial expression and voice data, and output is the recognized emotion. The device uses Microsoft Azure Face API and IBM Watson Speech to Text to estimate emotions.

[1197] Step 6:

[1198] The server suggests optimal interior design based on the recognized emotions. The input is the recognized emotions and generated visual information, while the output is the suggested interior design. The server adjusts the suggestions based on the user's emotions.

[1199] (Example 2)

[1200] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1201] Modern consumers demand personalized interior design suggestions tailored to their individual preferences and budgets, but traditional systems struggle to efficiently achieve this. There is also a need for easier access to available decorative items.

[1202] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1203] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images, means for planning interior decoration that matches the specified category, means for generating visual information of the planned interior decoration, means for analyzing information on decorative items included in the generated visual information, means for providing links to information sources where decorative items can be purchased based on the analysis results, and means for recognizing the user's emotions and suggesting interior decoration that corresponds to those emotions. This makes it possible to suggest personalized interior decoration that corresponds to the user's emotions and budget, and also makes it easier to access purchasable decorative items.

[1204] "Means for acquiring images of a room" refers to a device or method for receiving a photograph of a room taken by a user as digital data.

[1205] "Means for specifying a category" refers to a device or method for analyzing acquired images of a room and determining an appropriate interior genre or style based on those images.

[1206] "Means for planning interior decoration" refers to an apparatus or method for constructing an appropriate interior design based on a specified category.

[1207] "Means for generating visual information" refers to an apparatus or method for creating images or graphics to visually represent a planned interior decoration.

[1208] "Means for analyzing information on decorative items" refers to a device or method for identifying individual interior items included in the generated visual information and obtaining detailed information therefrom.

[1209] "Means of providing links to information sources" refers to devices or methods for providing users with links to online platforms or stores where the analyzed ornaments can be purchased.

[1210] "Means for recognizing user emotions" refers to a device or method for analyzing a user's facial expressions, tone of voice, operation speed, etc., to estimate the user's emotional state.

[1211] "Means of proposing interior decoration" refers to a device or method for presenting an optimal interior design to a user based on their perceived emotions and budget.

[1212] The following systems are conceivable as embodiments for carrying out this invention.

[1213] The server receives images of a room taken by the user using their device. The images are sent to the server as digital data, and the server analyzes the images using image recognition technology. This analysis determines the appropriate interior category based on the room's style and characteristics. For example, if the image contains a lot of wooden furniture, the "natural" category will be assigned.

[1214] Next, the server uses a generative AI model to plan the interior based on the specified category. The generative AI model is given specific instructions as prompts, such as "Please suggest an interior for a natural living room." Based on these prompts, the AI ​​model generates an interior image as visual information.

[1215] The generated interior image is sent from the server to the terminal and displayed to the user. The terminal analyzes each interior item included in the image and provides links to sources where each item can be purchased. For example, if the sofa shown in the image is available for purchase at a specific online store, a link to that store is presented to the user.

[1216] Furthermore, the device uses an emotion engine to analyze the user's facial expressions, tone of voice, and operation speed to estimate the user's emotions. Based on this emotion information, the server provides interior design suggestions that match the user's emotions. For example, if the user is expressing joy, the server will suggest an interior design with a bright and cheerful atmosphere.

[1217] In this way, users can easily find personalized interiors that suit their mood and budget.

[1218] The flow of the specific processing in Example 2 will be explained using Figure 19.

[1219] Step 1:

[1220] The user takes a picture of the room using their smartphone or camera. The captured image is saved as digital data on the device. The device then sends this image data to the server. The input is the image data of the room, and the output is the transmission of the image data to the server.

[1221] Step 2:

[1222] The server analyzes the received image data of the room. Using image recognition technology, it extracts features from the image and assigns an interior category. For example, if the image contains a lot of wooden furniture, the "natural" category will be assigned. The input is image data of the room, and the output is the assigned interior category.

[1223] Step 3:

[1224] The server uses a generative AI model to plan interiors based on a specified category. A prompt, such as "Please suggest a natural living room interior," is input to the generative AI model. The AI ​​model then generates interior images based on this prompt. The input consists of an interior category and a prompt, while the output is an interior image.

[1225] Step 4:

[1226] The server sends the generated interior image to the terminal. The terminal displays the received image to the user. The input is the interior image, and the output is the display of the image to the user.

[1227] Step 5:

[1228] The device analyzes each interior item included in the displayed interior image. Based on the analysis results, it generates and provides to the user links to sources where each item can be purchased. For example, if the sofa shown in the image is available for purchase at a specific online store, a link to that store will be displayed. The input is an interior image, and the output is a purchase link.

[1229] Step 6:

[1230] The terminal analyzes the user's facial expressions, tone of voice, and operation speed using an emotion engine to estimate the user's emotions. Based on this emotion information, the server provides interior design suggestions that correspond to the user's emotions. For example, if the user is expressing joy, it will suggest an interior design with a bright and cheerful atmosphere. The input is the user's emotion data, and the output is an interior design suggestion that corresponds to those emotions.

[1231] (Application Example 2)

[1232] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1233] Conventional interior design proposal systems have challenges in providing personalized suggestions that cater to users' emotions and budgets, and in offering insufficient means for users to easily purchase the suggested decorative items. Furthermore, because interior design suggestions are not based on users' emotions, it is difficult to improve user satisfaction.

[1234] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1235] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations suitable for the specified category, means for generating visual information of the planned interior decorations, and means for recognizing the user's emotions and suggesting interior decorations corresponding to those emotions. This makes it possible to suggest personalized interior decorations that match the user's emotions and budget, and further makes it possible to easily purchase the suggested decorative items.

[1236] "Method for acquiring images of the room" refers to a function that allows users to take photos of the room using their smartphones or cameras and import that image data into the system.

[1237] "Means for specifying a category and planning interior decoration suitable for the specified category" refers to a function that analyzes acquired images of a room, identifies the interior style and theme, and then selects and plans appropriate interior decoration based on that.

[1238] "Means for generating visual information of interior decoration" refers to a function that generates images and graphics to visually represent the planned interior decoration.

[1239] "A means of recognizing the user's emotions and proposing interior decorations that correspond to those emotions" refers to a function that analyzes the user's facial expressions and tone of voice to estimate their emotions and then proposes interior decorations that are appropriate for those emotions.

[1240] "Means of directing users to online platforms" refers to a function that provides links to online shopping sites where related decorative items can be purchased, based on the generated visual information of the interior decorations.

[1241] "A means of proposing interior decorations that fit the budget provided" refers to a function that selects and proposes interior decorations that are available for purchase within the budget range provided by the user.

[1242] The system for implementing this invention consists of a user terminal and a server. The user terminal is a portable information terminal such as a smartphone or tablet, equipped with a camera and microphone. The server provides computing resources for image processing, emotion recognition, and interior design proposals using generative AI models.

[1243] The user's device uses a camera to acquire images of the room. The acquired images are sent to a server, which analyzes the images using image recognition software (e.g., TensorFlow, OpenCV) and assigns a category. Based on the assigned category, the server uses a generative AI model (e.g., DALL-E, Stable Diffusion) to generate visual information for suitable interior decorations.

[1244] Furthermore, the user's device uses a microphone and camera to capture the user's facial expressions and voice tone, and sends this information to the server. The server uses emotion recognition software (e.g., Affectiva, Microsoft Azure Emotion API) to estimate the user's emotions and suggests interior decorations that correspond to those emotions.

[1245] The proposed interior decorations are displayed as visual information on the user's device, and links to online platforms where the related decorations can be purchased are provided. Based on the presented budget, the user can select interior decorations that are available within that budget.

[1246] For example, if a user takes a photo of their room with their smartphone and the app recognizes that the user wants to relax, the server will suggest calming color schemes for the interior and display a link to purchase the items. An example of a prompt would be, "Please suggest relaxing interior items that would suit this room."

[1247] The flow of a specific process in Application Example 2 will be explained using Figure 20.

[1248] Step 1:

[1249] The user uses the device's camera to capture images of the room. The captured images are sent from the device to the server. The input is the image data of the room, and the output is the transmission of the image data to the server.

[1250] Step 2:

[1251] The server analyzes the received image data using image recognition software (e.g., TensorFlow, OpenCV). The analysis results in a category based on the room's style and theme. The input is image data of the room, and the output is the specified category information.

[1252] Step 3:

[1253] The server generates visual information for suitable interior decorations using a generative AI model (e.g., DALL-E, Stable Diffusion) based on the specified category information. The input is category information, and the output is visual information for interior decorations.

[1254] Step 4:

[1255] The user uses the device's microphone and camera to capture their facial expressions and voice tone, and sends this data to the server. The input is the user's facial expression and voice tone data, and the output is the data transmission to the server.

[1256] Step 5:

[1257] The server analyzes the received facial expression and voice tone data using emotion recognition software (e.g., Affectiva, Microsoft Azure Emotion API) to estimate the user's emotion. The input is facial expression and voice tone data, and the output is the estimated emotion information.

[1258] Step 6:

[1259] The server suggests interior design elements suitable for the user based on estimated emotional information. The suggested interior design elements are displayed as visual information on the terminal. The input consists of emotional information and visual information of the interior design elements, and the output is the suggested display to the user.

[1260] Step 7:

[1261] Users can view visual information about interior decorations displayed on their device and proceed with a purchase by clicking on links to online platforms where related decorations can be purchased. The input is the user's selection, and the output is access to the online platform.

[1262] (Example 3)

[1263] Next, we will describe Embodiment 3 of Embodiment Example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1264] Modern consumers demand personalized interior design tailored to their individual emotions and budgets, but traditional systems struggle to efficiently achieve this. In particular, adjusting emotionally-based suggestions and selecting the most suitable decorative items within a given budget are difficult.

[1265] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

[1266] In this invention, the server includes means for acquiring images of a space using a camera, means for determining the category of the space based on the acquired images, means for planning decorative items that fit the determined category, means for analyzing the user's emotions, and means for adjusting the decorative item suggestions based on the analyzed emotions. This makes it possible to propose personalized spatial decorations that suit the user's emotions and budget.

[1267] A "photography device" is a device used to acquire images of a space, and includes devices such as cameras and smartphones.

[1268] A "space category" is a classification that indicates the style or theme of a space, determined based on the acquired images, and refers to genres such as "modern" or "classic."

[1269] "Decorative items" are items used to decorate a space, and include furniture, interior accessories, and works of art.

[1270] "Visual representation" refers to images or illustrations that visually show the planned arrangement and combination of decorative items.

[1271] "User emotions" refer to the psychological state a user exhibits when using the system, and are estimated from factors such as facial expressions, tone of voice, and speed of operation.

[1272] "Analyzing" is the process of analyzing data to extract specific information, and is done to understand emotions or the content of images.

[1273] "Adjusting the proposal" means changing the selection and placement of decorative items to suit the user's needs based on the analyzed information.

[1274] "Information provision" means providing links to online stores or retailers where the decorative items can be obtained, based on the generated visual representation.

[1275] "Financial information" refers to information about the budget provided by the user, and serves as a constraint in the selection of decorative items.

[1276] This system allows users to take pictures of their spaces and receive suggestions for decorative items that suit their budget and preferences. Users take photos of their rooms using a terminal and upload the images to the system. The server uses image analysis software to determine the category of the space. This analysis uses image recognition technology to identify categories such as "modern" or "classic" based on the room's color scheme and furniture style, for example.

[1277] Next, the server selects appropriate decorative items from the database based on the determined category and the user's budget. The server filters the price information of the decorative items to find the best combination within the budget. In this process, a generative AI model is used to generate visual representations of the decorative items. The generative AI model receives a prompt and generates an image that meets the specified conditions.

[1278] Furthermore, the server analyzes the user's facial expressions, tone of voice, and operation speed obtained from the terminal, and uses an emotion engine to estimate the user's emotions. Based on the analyzed emotions, the server adjusts its suggestions for accessories. For example, if the user wants to relax, it will suggest accessories in calming colors.

[1279] Finally, the server presents the user with a visual representation of the generated ornament and provides a link to the source of information where the ornament can be obtained. The user can click the link through their device and proceed with the purchase.

[1280] As a concrete example, if a user inputs "My budget is 50,000 yen, and I would like a relaxing living room," the server will suggest relaxing decorative items that can be purchased for under 50,000 yen and generate an image of them using a generation AI model. An example of a prompt would be, "Please suggest relaxing living room decorative items that can be purchased for under 50,000 yen." The specific processing flow in Example 3 will be explained using Figure 21.

[1281] Step 1:

[1282] The user takes a picture of the room using their device and uploads the image to the system. The input is the image of the room taken by the user. The output is the image data sent to the server. Specifically, the user uses their smartphone's camera function to take a picture of the entire room and uploads the image through the application.

[1283] Step 2:

[1284] The server inputs the received image data into image analysis software to determine the category of the space. The input is image data sent by the user. The output is the category information of the analyzed space. Specifically, the server uses an image recognition algorithm to analyze the color tones and furniture styles within the image and identify categories such as "modern" or "classic."

[1285] Step 3:

[1286] The server takes the determined category and user budget information as input and selects appropriate decorative items from the database. The input is the spatial category information and the user's budget information. The output is a list of decorative items that can be purchased within the budget. Specifically, the server executes a database query and filters out decorative items that fit the category and are within the budget.

[1287] Step 4:

[1288] The server uses a generative AI model to generate visual representations of selected ornaments. The input is a list of selected ornaments. The output is the generated visual representation of the ornaments. Specifically, the server inputs prompt statements into the generative AI model and generates images that meet the conditions.

[1289] Step 5:

[1290] The server uses an emotion engine to analyze the user's emotions, taking as input the user's facial expressions, voice tone, and operation speed obtained from the terminal. The input consists of the user's facial expression data and voice data. The output is the analyzed user emotion information. Specifically, the server uses a machine learning model to analyze the characteristics of facial expressions and voice and estimate the user's emotions.

[1291] Step 6:

[1292] The server adjusts accessory suggestions based on the analyzed emotional information. The input is the user's emotional information and a visual representation of the accessory. The output is the adjusted accessory suggestions. Specifically, the server changes the color and style of the accessories according to the user's emotions, providing the most suitable suggestions.

[1293] Step 7:

[1294] The server presents the user with a visual representation of the generated ornament and provides links to information sources where the ornament can be obtained. The input is a suggested, customized ornament. The output is the visual representation of the ornament and link information presented to the user. Specifically, the server sends the image and link to the terminal, through which the user can proceed with the purchase process.

[1295] (Application Example 3)

[1296] Next, we will describe application example 3 of form example 3. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1297] Modern consumers are seeking personalized interior design suggestions that cater to their individual emotions and budgets. However, traditional systems struggle to provide suggestions that consider user emotions and offer optimal solutions within a given budget. Furthermore, there is a lack of easy ways for consumers to purchase the suggested interior design items.

[1298] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

[1299] In this invention, the server includes means for acquiring images of a room, means for specifying a category based on the acquired images and planning interior decorations that match the specified category, means for generating visual information of the planned interior decorations, means for recognizing the user's emotions, means for suggesting interior decorations that correspond to the emotions based on the recognized emotions, means for suggesting interior decorations that match the budget provided by the user, and means for generating suggestions using a generative AI model. This makes it possible to suggest personalized interior decorations that correspond to the user's emotions and budget, and furthermore, makes it possible to easily purchase the suggested decorations.

[1300] "Method for acquiring images of the room" refers to a function that allows users to take photos of the room using their smartphones or cameras and import that image data into the system.

[1301] The "means for specifying categories" refer to a function that classifies the style and theme of interior decoration based on acquired images of rooms and selects the appropriate category.

[1302] "Means of planning interior decoration" refers to the function of selecting furniture and decorative items suitable for a room based on a specified category, and constructing the overall design.

[1303] "Means for generating visual information" refers to functions for visually representing the planned interior design as images or 3D models.

[1304] "Means of recognizing user emotions" refers to functions that analyze the user's facial expressions and tone of voice to estimate their emotional state.

[1305] "A means of proposing interior decoration that responds to emotions" refers to a function that selects and proposes interior decoration with an appropriate atmosphere and style based on the recognized emotions of the user.

[1306] "A means of proposing interior decoration based on a budget" refers to a function that selects furniture and decorative items that can be purchased within the budget provided by the user and proposes the most suitable interior decoration.

[1307] "Methods for generating proposals using generative AI models" refers to a function that utilizes artificial intelligence technology to automatically generate interior decoration proposals that meet the user's requests and conditions.

[1308] The system for implementing this invention starts by acquiring images of a room using the user's smartphone or camera and sending the image data to a server. The server uses image recognition technology to specify the room category from the acquired images and plans the interior decoration based on the specified category. The planned interior decoration is generated as visual information and presented to the user.

[1309] The server analyzes the user's facial expressions and voice tone using the smartphone's camera and microphone to recognize the user's emotions. Image recognition libraries (e.g., OpenCV) and emotion recognition APIs (e.g., Microsoft Azure Emotion API) are used for emotion recognition. Based on the recognized emotions, the server suggests interior decorations that correspond to those emotions.

[1310] Furthermore, based on the budget information provided by the user, the server selects interior decorations that can be purchased within the budget and provides optimal suggestions. A generative AI model (e.g., OpenAI GPT-3) is used to generate these suggestions. The generated suggestions are provided along with a link to an online platform for easy access by the user.

[1311] As a concrete example, consider a scenario where a user takes a photo of their living room with their smartphone and enters a budget of 50,000 yen. The server detects that the user has a cheerful expression and suggests bright and colorful interior decorations. The suggested items are displayed with links to online platforms.

[1312] An example of a prompt message is: "If the user's emotion is joy, please suggest bright and colorful interior items that can be purchased within a budget of 50,000 yen."

[1313] The flow of the specific processing in Application Example 3 will be explained using Figure 22.

[1314] Step 1:

[1315] The user takes a picture of the room using their smartphone camera. The captured image is sent from the device to the server. The input is the image data of the room, and the output is the image data sent to the server.

[1316] Step 2:

[1317] The server analyzes the received image data using image recognition technology (e.g., OpenCV) to specify the room category. The input is image data of the room, and the output is the specified category information. As a data processing step, features are extracted from the image and classified into categories.

[1318] Step 3:

[1319] The server plans the interior decoration based on the specified category. The planning process includes selecting appropriate furniture and decorations from a database. The input is category information, and the output is a list of the planned interior decorations.

[1320] Step 4:

[1321] The server generates visual information of the planned interior decoration. This visual information is represented as images or 3D models. The input is a list of interior decorations, and the output is the visual information. As a data operation, the selected items are visually arranged.

[1322] Step 5:

[1323] The user's emotions are recognized using their smartphone's camera and microphone. The server uses an emotion recognition API (e.g., Microsoft Azure Emotion API) to analyze the user's facial expressions and voice tone. The input is the user's facial expression data and voice data, and the output is the recognized emotion information.

[1324] Step 6:

[1325] The server suggests interior decorations that correspond to the recognized emotions. The input is emotional information, and the output is a suggestion for interior decorations that match those emotions. As a data processing step, a style appropriate to the emotion is selected.

[1326] Step 7:

[1327] The user enters budget information. The server selects interior decorations that can be purchased within the user's budget and provides optimal suggestions. The input is budget information, and the output is a suggestion of interior decorations that fit the budget. As part of the data calculation, items are selected considering price information.

[1328] Step 8:

[1329] The server generates the final proposal using a generative AI model (e.g., OpenAI GPT-3). The input consists of category information, sentiment information, and budget information, and the output is a final interior design proposal. Prompts are used to guide the AI ​​model in generating the proposal.

[1330] Step 9:

[1331] The server presents the generated proposals to the user and provides a link to the online platform. The input is the final interior design proposal, and the output is the display of the proposals to the user and a purchase link. Specifically, the server sends the proposal content to the user's device.

[1332] (Other examples)

[1333] Since this is the same as the specific processing described in the other embodiments of the first embodiment above, the explanation will be omitted.

[1334] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of ...

Claims

[Claim 1] Equipped with a processor, The aforementioned processor, Using a camera, we acquire images of the space. The acquired images are analyzed to extract features within the images, and the spatial category is determined. The system receives budget information from the user, searches the database based on the budget information, and selects items that can be purchased within the budget indicated by the budget information. By analyzing data on the user's facial expressions, tone of voice, or operation speed, an emotion identification model is used to estimate the user's emotions. A prompt for instructing the user to plan an interior design, which is the arrangement of interior furnishings to be placed in the space that conforms to a determined category, comprising a list of selected items and budget information, and generating a prompt that is adjusted according to the estimated user's emotions, and inputting it into a generating AI model to generate a visual representation of the interior design based on the budget and emotions. Based on the generated visual representation of the interior design, the elements within the visual representation are analyzed, compared with a product database, and links to online platforms where similar products corresponding to those elements can be purchased are generated and provided. system.

Images