System

Abstract

To provide a system for collectively providing efficient optimal interior coordination and optimal price merchandise purchase.SOLUTION: A system comprising: means for receiving floor plan data of a room; means for analyzing the received floor plan data to identify the structure and features of the room; means for generating a plurality of interior coordination plans based on the analysis results; means for listing products based on the generated interior coordination plans; means for searching for products at optimal prices from a plurality of shopping sites on the Internet for the listed products; means for calculating point benefits related to purchase from the search results; means for providing information including calculation results to a user; and means for generating a purchase link to a product selected by the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The spread of online shopping has made it easy for consumers to purchase a wide variety of interior goods, but the process of purchasing at the best price while considering the appropriate interior coordination remains cumbersome. Specifically, selecting furniture based on a room's layout, searching for that furniture on various shopping sites, and considering options such as point rewards is a significant burden. The objective of this invention is to provide a system that simplifies these complex procedures and efficiently provides optimal interior coordination and product purchases at the best price all in one place. [Means for solving the problem]

[0005] The present invention solves the above problem by providing a system including means for receiving room layout data, means for analyzing the received layout data and identifying the structure and characteristics of the room, means for generating a plurality of interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching for products at the best prices for the listed furniture from a plurality of shopping sites on the Internet, means for calculating point benefits associated with purchases from the search results, means for providing information including the calculation results to the user, and means for generating a purchase link to the product selected by the user.

[0006] "Room layout data" refers to information that indicates the layout and structure of a room, and includes drawings, photographs, CAD data, etc.

[0007] "Means of receiving" refers to the function of a device or program that inputs information such as room layout data from the user into the system.

[0008] "Means of analysis" refers to algorithms or programs that process the received data to understand the structure and characteristics of the room.

[0009] "Means for generating interior coordination plans" refers to algorithms or programs that create furniture arrangements and interior proposals that are optimal for the room's purpose and design based on the analysis results.

[0010] "Furniture listing means" refers to a program or process for selecting necessary furniture items based on the generated interior coordination plan.

[0011] "Best Price Search Method" refers to a program or algorithm that searches multiple shopping sites to find the lowest priced item for a listed furniture item.

[0012] "Means for calculating reward points" means a program or algorithm that calculates reward points or reward points associated with a purchase based on search results.

[0013] "Means for providing information to users" refers to the interface or program used to present calculated special offers and furniture product listings to users.

[0014] "Means for generating purchase links" means a program or process that creates a link to purchase a product selected by a user. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention relates to a system that proposes optimal interior coordination to users based on room layout data and also allows them to purchase the furniture at the optimal price. The specific operations performed in the processing of each step are described below.

[0037] System Configuration

[0038] The system mainly works between a server, a terminal, and a user. The user inputs the layout data of the room using the terminal, and the server uses that data to suggest interior coordination and provide optimal purchasing options.

[0039] Program processing

[0040] 1. Enter and submit room layout data

[0041] User: Takes a floor plan and photographs of the room, and uploads them to the system via a device.

[0042] Terminal: Sends the uploaded data to the server.

[0043] Server: Temporarily stores received data and prepares it for analysis.

[0044] 2. Analysis of room layout data

[0045] Server: Analyzes the received floor plan data using an AI algorithm to identify spatial characteristics such as room size, shape, available furniture placement areas, and the location of windows and doors.

[0046] Example: If a floor plan is divided into a living room, bedroom, and kitchen, the system automatically recognizes the area and shape of each room, as well as the space available for furniture placement.

[0047] 3. Creating an interior coordination plan

[0048] Server: Based on the analysis results, it generates multiple interior coordination plans according to the purpose of use of the space and the user's preferences.

[0049] Example: For a living room, suggestions include a Scandinavian-style furniture arrangement plan and a modern furniture arrangement plan.

[0050] 4. List your furniture items

[0051] Server: Generates a list of required furniture items based on the proposed interior coordination plan.

[0052] Example: Select key items for each category, such as sofas, dining tables, and beds.

[0053] 5. Find the best priced furniture

[0054] Server: Searches multiple online shopping sites for the best price for the listed furniture items.

[0055] Example: Search for "L-shaped sofa" on multiple shopping sites and determine the best product based on price, reviews, delivery terms, etc.

[0056] 6. Calculation of points rewards

[0057] Server: Calculates the points reward for each product based on the search results.

[0058] Example: Calculate the reward points when purchasing using a combination of Yahoo! Shopping and PayPay.

[0059] 7. Providing Information to the User Interface

[0060] Server: Sends calculated reward information and search results to the device.

[0061] Terminal: Displays proposed interior plans and product lists to the user through a user interface.

[0062] 8. Generate Purchase Link

[0063] User: Select the best from the proposed interior plans and product list.

[0064] Server: Generates a purchase link for the product selected by the user and sends it to the device.

[0065] Device: Show the purchase link to the user and assist them in the purchase process.

[0066] Specific examples

[0067] A user wants to redecorate their living room, so they upload a floor plan. The server analyzes the living room, recognizes its size and shape, and suggests furniture placement. It then searches the internet for the best prices for each furniture item and calculates the combination that maximizes point rewards. Finally, it generates a purchase link and presents it to the user, allowing them to easily purchase the furniture.

[0068] This system allows users to effortlessly coordinate their interiors and purchase furniture economically all in one place.

[0069] The processing flow will be explained below.

[0070] Step 1:

[0071] User: Take a floor plan or photo of the room and upload it to the device.

[0072] Step 2:

[0073] Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[0074] Step 3:

[0075] Server: Temporarily stores the received floor plan data and prepares it for data analysis.

[0076] Step 4:

[0077] Server: Analyzes stored floor plan data using AI algorithms to identify spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[0078] Step 5:

[0079] Server: Generates multiple interior coordination plans based on the analysis results, proposing plans that take into account style, color scheme, furniture placement, etc.

[0080] Step 6:

[0081] Server: Generates a list of required furniture items based on each interior coordination plan.

[0082] Step 7:

[0083] Server: Initiates a search for the listed furniture items using APIs and scraping technology to find the best prices from multiple shopping sites.

[0084] Step 8:

[0085] Server: Retrieves and lists the cheapest price information (price, retailer, ratings, availability, etc.) found for each furniture item.

[0086] Step 9:

[0087] Server: Based on the search results, calculate the points rewards for each product (such as Yahoo! Shopping or PayPay points).

[0088] Step 10:

[0089] Server: Sends the interior coordination plan and furniture list including the calculation results to the terminal.

[0090] Step 11:

[0091] Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through an interface.

[0092] Step 12:

[0093] User: Review the proposed interior coordination plan and furniture list and select the one they like.

[0094] Step 13:

[0095] Terminal: Accepts the user's selection and resends the information to the server.

[0096] Step 14:

[0097] Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[0098] Step 15:

[0099] Device: Show the purchase link to the user and assist them in the purchase process.

[0100] Step 16:

[0101] User: Use the provided purchase link to purchase furniture through a shopping site or PayPay.

[0102] Step 17:

[0103] Server: Follow up with the user after the purchase by sending them a confirmation of the purchase and tracking information.

[0104] Example 1

[0105] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] Conventional interior coordination systems lacked the functionality to suggest optimal furniture placement to users. Furthermore, providing information on how to purchase furniture at the best price and calculating point rewards associated with purchases were often done manually, which was time-consuming for users. Furthermore, the systems lacked the means to safely handle users' personal information, leaving them at risk of data theft and unauthorized access.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0108] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching multiple online shopping sites for products at the best price for the listed furniture, means for calculating points related to purchases from the search results, means for providing the user with information including the calculation results, means for generating a purchase link to the product selected by the user, and means for encrypting and transmitting data to protect the user's privacy. This allows the user to effortlessly coordinate an optimal interior and purchase furniture economically all in one place.

[0109] "Room layout data" refers to data provided by users that includes information such as the dimensions and shape of the room, the area in which furniture can be placed, and the positions of windows and doors.

[0110] The "receiving means" is a system or device for receiving room layout data from a user.

[0111] "Means for analysis" refers to a system or device that analyzes the received room layout data using an AI algorithm or the like to identify the structure and characteristics of the room.

[0112] A "means for identifying features" is a system or device that has the ability to extract important spatial information from the analysis results, such as the size and shape of the room, the area where furniture can be placed, and the location of windows and doors.

[0113] An "interior coordination plan" is data that shows furniture placement and design plans proposed based on the user's preferences and room characteristics.

[0114] The "generating means" is a system or device for creating a plurality of interior coordination plans based on the analysis results.

[0115] The "listing means" is a system or device having a function for creating a list of necessary furniture and decorative items based on the generated interior coordination plan.

[0116] The "means for searching for products at the best price" is a system or device that has the function of searching for the most favorable priced product for the listed furniture from multiple online shopping sites.

[0117] The "means for calculating point benefits" is a system or device that has the function of calculating the total points redeemed based on the point redemption information provided by multiple shopping sites on the Internet based on search results.

[0118] The "means for providing information including calculation results to users" refers to a system or device for notifying users of search results including point redemption information and visually displaying them.

[0119] The "means for generating a purchase link" refers to a system or device for creating a link that allows a user to directly purchase the product selected by the user on an online shopping site.

[0120] "Means for encryption and transmission" refers to a system or device that encrypts data such as floor plan data and user information and communicates them securely in order to protect user privacy.

[0121] This invention relates to a system that proposes optimal interior coordination to users based on room layout data and allows them to purchase the furniture at the optimal price. This system functions in cooperation with a server, terminals, and users.

[0122] System Configuration

[0123] The user inputs the layout data of the room using a terminal, and the server uses that data to propose an interior coordination plan and also provides optimal purchasing options. In terms of specific hardware, the user's terminal can be a smartphone, tablet, or PC, while the server uses a high-performance computer system.

[0124] Entering and Submitting Data

[0125] User

[0126] Users take photos of the room floor plan and other images using the camera on their smartphone or PC, and upload the data to the system using the following prompts:

[0127] "I uploaded the living room floor plan."

[0128] "I sent you the bedroom floor plan."

[0129] Terminal

[0130] The terminal temporarily stores the data entered by the user and transmits it to the server using a secure communication protocol (HTTPS).

[0131] server

[0132] The server temporarily stores the received data and checks the data for consistency and format. If any errors are found, the user is notified and asked to re-upload.

[0133] Data analysis

[0134] server

[0135] The server uses AI algorithms (e.g., Convolutional Neural Networks) to analyze the data it receives, identifying important spatial features such as the room's size, shape, available furniture area, and the location of windows and doors.

[0136] As a specific example, if a floor plan is divided into a living room, bedroom, and kitchen, the area and shape of each room, as well as the space available for furniture placement, are automatically recognized using libraries such as HTML5 Canvas and OpenCV.

[0137] Coordination plan generation

[0138] server

[0139] The server generates multiple interior coordination plans based on the above analysis results, tailored to the user's preferences and lifestyle. Using a specific generation AI model, the server generates plans using the following prompts:

[0140] "Please suggest a Scandinavian-style interior design for the living room."

[0141] "Generate a modern bedroom furniture arrangement plan."

[0142] The generated plans are offered for selection by the user.

[0143] Listing furniture items

[0144] server

[0145] Based on the plan generated, create a list of the furniture items you'll need, including items in each major category, such as a sofa, dining table, and bed.

[0146] Find the best price

[0147] server

[0148] The server searches multiple online shopping sites in real time for the best price for the listed furniture, taking into account not only price but also ratings, delivery terms, and other factors.

[0149] Points Reward Calculation

[0150] server

[0151] Based on the search results, the points rewards for each product are calculated. For example, the total points are calculated based on the point reward rates on Yahoo! Shopping and other online platforms.

[0152] Providing information

[0153] server

[0154] Information including the calculation results is sent to the terminal, which then visually displays it so that the user can easily understand it.

[0155] Generate purchase links

[0156] User

[0157] The user selects the most suitable item from the proposed interior plans and product list.

[0158] server

[0159] The server generates a purchase link for the product selected by the user and sends it to the terminal, which displays the link to the user and assists in the purchase process.

[0160] Data encryption

[0161] Terminals and Servers

[0162] To protect user privacy, all data transmitted is encrypted using Transport Layer Security (TLS), which protects user information from data theft and unauthorized access.

[0163] As described above, this system minimizes the user's effort and enables optimal interior coordination and economical furniture purchases.

[0164] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0165] Step 1:

[0166] input

[0167] User: Take a floor plan or photo of a room.

[0168] Users: Upload data via a dedicated application or web portal.

[0169] output

[0170] Device: Temporarily stores floor plans and photo data.

[0171] Terminal: Sends data to the server using a secure communication protocol (HTTPS).

[0172] Specific actions

[0173] Users take a picture of the room floor plan using the camera on their smartphone or PC.

[0174] The captured data is saved in the application and sent to the server.

[0175] Step 2:

[0176] input

[0177] Terminal: Receives floor plan data from the user.

[0178] Server: Receives floor plan data sent from the device.

[0179] output

[0180] Server: Temporarily stores the received data and checks its structural integrity.

[0181] Server: If the data is in the correct format, prepare it for analysis. If it is not, send an error notification to the user.

[0182] Specific actions

[0183] The server checks the integrity of the data and notifies the user if there are any discrepancies.

[0184] If the data format is correct, proceed to the next analysis step.

[0185] Step 3:

[0186] input

[0187] Server: Received floor plan data

[0188] output

[0189] Server: Analyzes floor plan data and identifies room size, shape, available furniture placement area, and window and door locations.

[0190] Specific actions

[0191] The server analyzes the floor plan data using an AI algorithm (e.g., Convolutional Neural Network).

[0192] Identify room boundaries, dimensions, and functional areas and store them in a database.

[0193] Step 4:

[0194] input

[0195] Server: Analysis results (room size, shape, furniture placement area, etc.)

[0196] output

[0197] Server: Multiple interior coordination plans based on user preferences

[0198] Specific actions

[0199] The server uses a generative AI model to generate a coordination plan based on the prompt sentence.

[0200] Example: "Generate a Scandinavian furniture arrangement plan for my living room."

[0201] Organize the proposed plans and select the most suitable one.

[0202] Step 5:

[0203] input

[0204] Server: Generated interior coordination plan

[0205] output

[0206] Server: List of required furniture items

[0207] Specific actions

[0208] The server lists the necessary furniture items based on the coordination plan.

[0209] Select key items for each category, such as sofas and dining tables.

[0210] Step 6:

[0211] input

[0212] Server: List of furniture items

[0213] output

[0214] Server: A list of products with the best prices from multiple shopping sites on the Internet

[0215] Specific actions

[0216] The server crawls multiple shopping sites and compares prices.

[0217] Example: Search for "L-shaped sofa" across multiple sites to identify the best products.

[0218] Product ratings and delivery terms are also taken into consideration.

[0219] Step 7:

[0220] input

[0221] Server: Price comparison results

[0222] output

[0223] Server: Calculation result of point benefits for each product

[0224] Specific actions

[0225] The server calculates the point rewards for each product based on the point redemption rate.

[0226] Example: Calculating Yahoo! Shopping reward points.

[0227] Step 8:

[0228] input

[0229] Server: Calculated point reward information

[0230] Terminal: Information received from the server

[0231] output

[0232] Terminal: Displays interior plans and product lists in a user-friendly format

[0233] Specific actions

[0234] The terminal retrieves the calculated reward information and displays it in a graphical interface.

[0235] Product images, price information, point redemption information, and more can be viewed at a glance.

[0236] Step 9:

[0237] input

[0238] User: Select the furniture item they want from the list of suggested items

[0239] output

[0240] Server: Purchase link for selected product

[0241] Specific actions

[0242] The server generates a purchase link for the furniture item selected by the user.

[0243] Send a purchase link to the device to allow users to easily make a purchase.

[0244] By following the above steps, users can efficiently and easily coordinate their interiors optimally and purchase furniture economically.

[0245] (Application example 1)

[0246] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0247] Currently, when coordinating interiors, it is difficult to select and arrange the optimal furniture to fit the floor plan. Furthermore, there is a lack of ways to purchase furniture at the best price or to virtually check what the interior will look like after purchase. This requires users to spend a lot of time and effort, making it difficult to achieve effective interior coordination.

[0248] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0249] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching multiple online shopping sites for products at the best prices for the listed furniture, means for calculating point benefits associated with purchases from the search results, means for providing the user with information including the calculation results, means for generating purchase links to products selected by the user, means for the user to 3D scan the floor plan data, and means for confirming the interior coordination plan in a virtual space using the 3D scanned data. This allows the user to efficiently and economically realize optimal interior coordination and virtually confirm it.

[0250] "Room layout data" is information that indicates the layout, size, and shape of a physical space such as a user's residence or office.

[0251] "Means for receiving" refers to devices or software that have the functionality to allow the system to acquire data sent from the user.

[0252] "Means for analysis" refers to algorithms or software that analyze the received data and identify the structure and characteristics of the space.

[0253] An "interior coordination plan" is a specific proposal for the layout and design of furniture and decorations to be placed in a room.

[0254] The "listing means" refers to a device or software that has the function of selecting the necessary furniture and products based on the interior coordination plan.

[0255] "Internet shopping site" refers to a website where users can search for and purchase products online.

[0256] A "means for searching for products at the best price" is an algorithm or software that searches multiple shopping sites for products that meet your criteria.

[0257] The "means for calculating point benefits" refers to a device or software that has the function of calculating points or discounts that can be obtained at the time of purchase.

[0258] "Means for providing information to users" refers to the interface or software for displaying calculation results and proposals to users.

[0259] The "means for generating a purchase link" refers to a device or software that has the function of creating a link for purchasing the product selected by the user.

[0260] "3D scanning means" refers to devices and software for digitizing physical spaces as three-dimensional data.

[0261] "Means for checking in a virtual space" refers to devices or software that have the function of allowing users to visually check interior coordination plans using three-dimensional models.

[0262] The present invention relates to a system that allows a user to propose optimal interior coordination based on room layout data, and detailed embodiments thereof will be described.

[0263] System Configuration

[0264] The system mainly functions between the server, the terminal, and the user. The system receives room layout data and 3D scan data from the user and proposes interior coordination in the virtual space. Specifically, it works as follows:

[0265] Hardware and Software

[0266] Hardware:

[0267] A smartphone or head-mounted display (e.g., Oculus Quest 2)

[0268] Server (Cloud server or on-premise server)

[0269] software:

[0270] Unity, Vuforia (AR technology), OpenCV

[0271] Python and Flask (server-side processing)

[0272] MongoDB (data storage)

[0273] Processing flow

[0274] Data reception and analysis

[0275] 1. User:

[0276] Use a smartphone or head-mounted display to 3D scan the layout of a room.

[0277] The acquired data is sent from the terminal to the server.

[0278] 2. Server:

[0279] The server analyzes the received 3D scan data using OpenCV to determine the size, shape, and area where furniture can be placed in the room.

[0280] The analysis results are saved in MongoDB.

[0281] Generate interior coordination plans

[0282] 3. Server:

[0283] Based on the analysis results, a Python script is used to generate multiple interior coordination plans, such as Scandinavian and modern styles.

[0284] Check in virtual space

[0285] 4. User:

[0286] Using Unity and Vuforia, participants can visually check 3D models of proposed interior coordination plans in a virtual space using a head-mounted display or smartphone.

[0287] Product listing and price search

[0288] 5. Server:

[0289] The system creates a list of furniture items based on the generated interior coordination plan and uses a Python script to search multiple online shopping sites for the best prices.

[0290] Calculate points rewards for each product based on search results.

[0291] Purchase links and information

[0292] 6. Server:

[0293] The search results and calculated point reward information are sent to the terminal and displayed to the user.

[0294] A purchase link is generated for the product selected by the user and sent to the terminal.

[0295] 7. Users:

[0296] Check the proposed interior plans and product list and select the most suitable products.

[0297] Easily purchase products through our purchase links.

[0298] Prompt Sentence Examples

[0299] The invention provides a virtual interior coordination assistant application that allows users to send scanned data of a room to a server and easily coordinate the optimal interior and compare prices.

[0300] As a concrete example, a user considers redecorating their bedroom and scans the room using a head-mounted display. The system analyzes the size and shape of the room and proposes interior plans in both Scandinavian and modern styles. The user can then view the virtual 3D models and purchase furniture that suits their tastes at the best price.

[0301] This system allows users to efficiently coordinate their interiors optimally and purchase furniture at economical prices.

[0302] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0303] Step 1:

[0304] Users use a smartphone or head-mounted display to 3D scan the layout of a room, and the scanned data is then sent from the device to a server.

[0305] Input: 3D scan of a room.

[0306] Processing: The device receives the 3D scan data taken by the user, converts it into the appropriate format, and sends it to the server.

[0307] Output: 3D scan data sent to the server.

[0308] Step 2:

[0309] The server analyzes the received 3D scan data to determine the size, shape, and available furniture area of ​​the room using OpenCV.

[0310] Input: 3D scan data.

[0311] Processing: Using OpenCV, the server analyzes the 3D scan data and extracts spatial features (size, shape, furniture placement area), applying image processing techniques such as edge detection and shape analysis.

[0312] Output: Data about the room size, shape, and available furniture area.

[0313] Step 3:

[0314] The server generates multiple interior coordination plans based on the analysis results, using a Python script.

[0315] Input: Data about the room size, shape, and available furniture area.

[0316] Processing: Using a Python script, multiple interior coordination plans (e.g., Scandinavian style, modern style) are generated based on the analysis results. Various plans are created by referencing past design data and user preferences.

[0317] Output: Interior coordination plan data.

[0318] Step 4:

[0319] Using Unity and Vuforia, users can visually view 3D models of proposed interior coordination plans in a virtual space using a head-mounted display or smartphone.

[0320] Input: Interior coordination plan data.

[0321] Processing: Using Unity and Vuforia, the interior coordination plan is created as a 3D model and displayed in a user interface, allowing users to visually inspect the model and perform operations such as rotating and scaling.

[0322] Output: A virtual interior coordination that can be visually confirmed by the user.

[0323] Step 5:

[0324] The server lists furniture items based on the interior coordination plan and uses a Python script to search multiple shopping sites on the Internet for the best priced items.

[0325] Input: Interior coordination plan data.

[0326] Processing: The server crawls the web for the listed furniture items, collects information from multiple shopping sites, and compares and selects the best options based on price, reviews, delivery terms, etc.

[0327] Output: A list of the best purchasing options for each furniture item.

[0328] Step 6:

[0329] The server calculates the points reward for each product based on the search results.

[0330] Enter: the best buying options list.

[0331] Processing: The server obtains the point reward information of each shopping site and calculates the number of points that can be earned at the time of purchase. This is done to calculate the economic efficiency of the purchase.

[0332] Output: A list of optimal purchase options with point rewards.

[0333] Step 7:

[0334] The server generates the calculation results and a link to purchase the furniture, and sends them to the device. The user can then use their smartphone or head-mounted display to check the interior plan and product list and select the most suitable product. The product can then be easily purchased using the purchase link.

[0335] Enter: a list of the best purchasing options, including points rewards.

[0336] Processing: Organize the calculated reward information and product list, generate data to display in the user interface, generate a purchase link for the product selected by the user, and send it to the device.

[0337] Output: Interior plans and product list with purchase links.

[0338] This series of processes allows users to efficiently and economically create optimal interior coordination and easily purchase furniture while checking it virtually.

[0339] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0340] This invention combines an emotion engine with a system that proposes optimal interior coordination to users based on room layout data and then allows them to purchase that furniture at the optimal price. The emotion engine recognizes the user's emotional state and makes suggestions based on that.

[0341] System Configuration

[0342] The system mainly works between the server, the terminal, and the user. The user inputs the room layout data using the terminal, and the server proposes interior coordination based on that data and the user's emotional data, and provides optimal purchasing options.

[0343] Program processing

[0344] Enter and submit room layout data

[0345] 1. User: Take a floor plan or photo of the room and upload it to the device.

[0346] 2. Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[0347] 3. Server: Temporarily stores the received data and prepares it for analysis.

[0348] Analysis of room layout data

[0349] 4. Server: The stored floor plan data is analyzed using AI algorithms to identify spatial characteristics such as room size, shape, furniture placement area, and window and door locations.

[0350] Generate interior coordination plans

[0351] 5. Server: Based on the analysis results, it generates multiple interior coordination plans, taking into account style, color scheme, furniture placement, etc.

[0352] Emotion data input and analysis

[0353] 6. User: Emotional data can be input to the device using facial recognition via a webcam, text input, or voice input.

[0354] 7. Terminal: Receives the user's emotion data and sends it to the server.

[0355] 8. Server: Analyzes the received emotion data using the emotion engine and identifies the user's emotional state.

[0356] Generate personalized proposals

[0357] 9. Server: Selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, it will suggest furniture with calming colors and soft textures.

[0358] Listing furniture items

[0359] 10. Server: Generates a list of required furniture items based on each interior coordination plan.

[0360] Find the best priced furniture

[0361] 11. Server: Initiates a search for the listed furniture items using APIs and scraping techniques to find the best prices from multiple shopping sites.

[0362] 12. Server: Retrieves and lists the cheapest price information (price, retailer, rating, availability, etc.) found for each furniture item.

[0363] Points Reward Calculation

[0364] 13. Server: Based on the search results, calculates the points rewards for each product (such as points redemption on shopping sites or payment platforms).

[0365] Providing information to the user interface

[0366] 14. Server: Sends the calculated reward information and search results to the device.

[0367] 15. Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through the interface.

[0368] Generate purchase links

[0369] 16. User: Review the proposed interior coordination plan and furniture list and select the one they like best.

[0370] 17. Terminal: Accepts the user's selection and resends the information to the server.

[0371] 18. Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[0372] 19. Device: Display the purchase link to the user and assist with the purchase process.

[0373] Purchasing and Follow-up

[0374] 20. User: Uses the provided purchase link to purchase furniture through a shopping site or payment platform.

[0375] 21. Server: Follow up on the purchase by sending the user a confirmation of the purchase and tracking information.

[0376] Specific examples

[0377] A user wants to redecorate their living room and uploads a floor plan. The server analyzes the living room, recognizes its size and shape, and suggests furniture placement. The user then provides emotional data via webcam, revealing that they are looking for a relaxing space. Based on this information, the server suggests furniture with calming color schemes and interior design that creates a relaxing effect. The server searches multiple shopping sites for the best prices and point rewards for the furniture and provides them to the user. Once the user selects the furniture, a purchase link is generated to help them easily complete the purchase process.

[0378] This system allows users to coordinate their interiors to best suit their emotional state and makes hassle-free and economical furniture purchases all in one go.

[0379] The processing flow will be explained below.

[0380] Step 1:

[0381] User: Take a floor plan or photo of the room and upload it to the device.

[0382] Step 2:

[0383] Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[0384] Step 3:

[0385] Server: Temporarily stores the received floor plan data and prepares it for data analysis.

[0386] Step 4:

[0387] Server: Analyzes stored floor plan data using AI algorithms to identify spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[0388] Step 5:

[0389] Server: Generates multiple interior coordination plans based on the analysis results, proposing plans that take into account style, color scheme, furniture placement, etc.

[0390] Step 6:

[0391] User: Enters emotion data into the device using face recognition via a webcam, text input, or voice input.

[0392] Step 7:

[0393] Terminal: Receives the user's emotion data and sends the emotion data to the server.

[0394] Step 8:

[0395] Server: Analyzes the received emotional data using the emotion engine and identifies the user's emotional state. If the user wants to relax, it will be labeled as "Relaxed," and if the user wants to be energized, it will be labeled as "Energy."

[0396] Step 9:

[0397] Server: Selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, the server will suggest furniture with calming colors and soft textures.

[0398] Step 10:

[0399] Server: Generates a list of required furniture items based on each interior coordination plan.

[0400] Step 11:

[0401] Server: Initiates a search for the listed furniture items using APIs and scraping technology to find the best prices from multiple shopping sites.

[0402] Step 12:

[0403] Server: Retrieves and lists the cheapest price information (price, retailer, ratings, availability, etc.) found for each furniture item.

[0404] Step 13:

[0405] Server: Based on the search results, calculates the reward points for each product (such as points redemption from shopping sites or payment platforms).

[0406] Step 14:

[0407] Server: Sends the interior coordination plan and furniture list including the calculation results to the terminal.

[0408] Step 15:

[0409] Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through an interface.

[0410] Step 16:

[0411] User: Review the proposed interior coordination plan and furniture list and select the one they like.

[0412] Step 17:

[0413] Terminal: Accepts the user's selection and resends the information to the server.

[0414] Step 18:

[0415] Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[0416] Step 19:

[0417] Device: Show the purchase link to the user and assist them in the purchase process.

[0418] Step 20:

[0419] User: Purchases furniture through a shopping site or payment platform using the provided purchase link.

[0420] Step 21:

[0421] Server: Follow up with the user after the purchase by sending them a confirmation of the purchase and tracking information.

[0422] Example 2

[0423] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0424] Conventional interior coordination suggestion systems have difficulty making personalized suggestions that take into account the user's emotional state. Furthermore, searching for furniture prices and presenting point benefits is time-consuming, and the system provides insufficient support for users to make optimal choices. This makes it difficult to achieve interior coordination that satisfies users and to efficiently purchase furniture.

[0425] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for acquiring and analyzing the user's emotional state, means for selecting an appropriate interior coordination plan based on the acquired emotional state, means for listing furniture based on the generated interior coordination plan, means for searching multiple shopping sites on the Internet for products at the best prices for the listed furniture, means for calculating point benefits related to purchases from the search results, means for providing the user with information including the calculation results, and means for generating purchase links to products selected by the user. This allows the server to suggest optimal interior coordination based on the user's emotional state, enabling efficient and economical furniture purchases.

[0426] "Room layout data" is numerical and graphical information that indicates the shape and dimensions of a room, the area in which furniture can be placed, the positions of windows and doors, and so on.

[0427] An "interior coordination plan" is a proposal for the interior design of a room, including the style, color scheme, furniture placement, and decorative items.

[0428] "Emotional state" is information that indicates the user's mood and state of mind, and is primarily obtained through facial recognition, voice analysis, text input, etc.

[0429] A "furniture list" is a list of necessary furniture items based on a specific interior coordination plan, including detailed information about each item.

[0430] The "best price" is the lowest price obtained from multiple shopping sites on the Internet for the furniture item specified by the user.

[0431] "Point benefits" are points or discount information provided by shopping sites or payment platforms that are given back to users when they make a purchase.

[0432] A "purchase link" is a direct URL or hyperlink to an online shopping site that allows a user to purchase a specified furniture item.

[0433] This invention is a system that proposes optimal interior coordination to users based on room layout data. This system makes personalized suggestions based on the user's emotional state, and also allows the user to purchase furniture at the optimal price based on those suggestions.

[0434] System Configuration

[0435] This system mainly functions between a server, a terminal, and a user. The user inputs room layout data using the terminal, and the server proposes interior coordination based on that data and the user's emotional data, and provides optimal purchasing options.

[0436] Hardware and software used

[0437] Devices: Smartphone, tablet, PC (with webcam)

[0438] Server: Cloud server (e.g. Amazon Web Services)

[0439] AI algorithm: TENSORFLOW (registered trademark) (for image analysis)

[0440] Emotion analysis engine: Emotion API (face recognition, voice analysis)

[0441] Database: SQL database (for managing customer information and furniture lists)

[0442] API: Web scraping tool (for product price search)

[0443] Program processing

[0444] Enter and submit room layout data

[0445] The user uses the device to take floor plans and photos of the room and uploads them to the device through a dedicated application. The device receives the data uploaded by the user and sends it to the server.

[0446] Analysis of room layout data

[0447] The server analyzes the received floor plan data using an AI algorithm (e.g., TensorFlow) to identify spatial characteristics such as room size, shape, and available furniture placement areas, as well as the location of windows and doors.

[0448] Generate interior coordination plans

[0449] The server generates multiple interior coordination plans based on the analysis results, including different styles, color schemes, and furniture placements.

[0450] Emotion data input and analysis

[0451] Users can input emotion data into their devices using facial recognition via a webcam, text input, or voice input. The device receives the user's emotion data and sends it to a server. The server then uses an emotion analysis engine (e.g., Emotion API) to identify the user's emotional state.

[0452] Generate personalized proposals

[0453] The server selects the most appropriate interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking to relax, it will suggest furniture with a calm color scheme and soft textures.

[0454] Listing furniture items

[0455] The server generates a list of necessary furniture items based on the selected interior plan, including detailed information about each piece (size, design, material, etc.).

[0456] Find the best priced furniture

[0457] The server searches for the best priced products for the listed furniture items from multiple shopping sites using APIs and scraping technology, and obtains the best prices and ratings for the furniture based on the search results.

[0458] Points Reward Calculation

[0459] The server calculates the points rewards (points redeemed by shopping sites and payment platforms) for each product, allowing users to make the most economical overall choice.

[0460] Providing information to the user interface

[0461] The server transmits the calculated reward information and search results to the terminal, which displays the information to the user through an interface.

[0462] Generate purchase links

[0463] The user checks the presented interior coordination plan and furniture list and selects the furniture they want. The terminal retransmits the user's selection to the server. The server generates a purchase link for the furniture item selected by the user and transmits it again to the terminal. The terminal displays the purchase link to the user and supports the purchase procedure.

[0464] Purchasing and Follow-up

[0465] The user uses the provided purchase link to purchase the furniture through a shopping site or payment platform, and the server then sends the user a confirmation of the purchase and tracking information as follow-up after the purchase.

[0466] Specific operation example

[0467] For example, if a user is thinking about redecorating their living room, they first upload a floor plan. The server analyzes the floor plan and recognizes the size and shape of the living room. Next, the user provides emotional data via webcam, and it is determined that they are looking for a relaxing space. Based on this information, the server suggests calming color schemes and interior design that has a relaxing effect. It also searches multiple shopping sites for the best prices and point rewards for furniture and provides them to the user. The server then generates a purchase link for the furniture selected by the user, facilitating a simple purchasing process.

[0468] Prompt Sentence Examples

[0469] I'm thinking of redecorating my living room. Please suggest the best interior design based on this floor plan. I want to create a relaxing space.

[0470] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0471] Step 1:

[0472] User: Takes a photo or a floor plan of the room and uploads it to the device via a dedicated application. The request includes the floor plan and photo data. For example, a user takes a photo of the living room floor plan with their smartphone and presses the "upload" button.

[0473] Terminal: Receives floor plan data uploaded by the user, checks the data format (JPEG or PNG), and sends it to the server. Once the data format check is complete, it is sent to the server.

[0474] Specific operation: When the user presses the "Upload floor plan" button, the image data of the floor plan is saved on the device and transferred to the server.

[0475] Step 2:

[0476] Server: The received floor plan data is temporarily stored in cloud storage (e.g., Amazon S3), and then an AI algorithm (TensorFlow) is invoked to analyze it.

[0477] Input: Received image data of floor plans and photos

[0478] Output: Analytical data including room size, shape, available furniture area, and spatial characteristics such as window and door locations

[0479] Specific operation: The server passes the stored floor plan to an AI algorithm, which performs edge detection and object recognition. The analysis results are then stored in a database.

[0480] Step 3:

[0481] Server: Based on the analysis results, the server executes the interior coordination plan generation program. Here, it references several interior style templates from the database and selects a plan that suits the user's room.

[0482] Input: Room analysis data

[0483] Output: Multiple interior coordination plans (style, color scheme, furniture placement)

[0484] Specific operation: Based on the analysis data, multiple optimal templates are extracted and images and text information for presentations are generated.

[0485] Step 4:

[0486] User: Activate the device's webcam and use the facial recognition system to input emotion data, or choose to input text or voice.

[0487] Terminal: Receives the user's emotional data (facial images and voice data), sends the data to the Emotion API for real-time analysis.

[0488] Input: User emotion data (image, voice, text)

[0489] Output: User's emotional state (relaxed, stressed, excited, etc.)

[0490] What it does: The webcam captures the user's face and sends the data in real time to the Emotion API to analyze their emotional state.

[0491] Step 5:

[0492] Server: Based on the analysis results of the emotion engine, the server selects the most suitable interior coordination plan. For example, if the user is looking for relaxation, the server will suggest a calm color scheme and soft furniture.

[0493] Input: Emotional state data and initial analysis data

[0494] Output: Optimal interior coordination plan based on emotional state

[0495] Specific operation: The system automatically adjusts the priority of interior plans based on emotional data, and selects and displays the top plans.

[0496] Step 6:

[0497] Server: Generates a list of furniture items based on the selected interior coordination plan. The list includes detailed information about the items (size, design, material, etc.).

[0498] Input: Interior coordination plan

[0499] Output: A list of furniture items

[0500] Specific operation: Extract relevant furniture information from the database and compile it in list form.

[0501] Step 7:

[0502] Server: For the listed furniture items, search for the best price from multiple shopping sites using APIs and scraping technology. Collect price information and ratings.

[0503] Input: List of furniture items

[0504] Output: Optimal price and rating information for each furniture item

[0505] Specific operation: Use a web scraping tool or API to search for and collect information on relevant items on each shopping site.

[0506] Step 8:

[0507] Server: Calculates the points rewards for each product. Calculates the actual price based on the point reward information from the shopping site or payment platform.

[0508] Input: Furniture item price and special offer information

[0509] Output: Details of point redemption as calculation result

[0510] Specific operation: Calculates the point redemption rate and displays the actual payment amount and redemption points.

[0511] Step 9:

[0512] Server: Sends calculation results and analysis data to the terminal, allowing the user to check the presented information.

[0513] Terminal: Displays interior coordination plans, furniture lists, and point benefit information to the user through an interface.

[0514] Input: Calculation results and analysis data

[0515] Output: Information displayed in the user interface

[0516] Specific operation: The GUI on the terminal displays information in an easy-to-understand format and presents options.

[0517] Step 10:

[0518] User: Check the proposed interior coordination plan and furniture list and select the furniture you want.

[0519] Terminal: Accepts the user's selection and retransmits it to the server.

[0520] Input: User selection information

[0521] Output: Data for selected furniture items

[0522] Specific operation: When the user presses the selection button, the selection information is sent to the server.

[0523] Step 11:

[0524] Server: Generates a purchase link for the furniture item selected by the user and sends it back to the terminal.

[0525] Device: Displays the purchase link to the user and assists them in the purchase process.

[0526] Input: Selection information

[0527] Output: Purchase link for furniture item

[0528] Specific operation: The server generates a purchase link and sends it to the device, where it is displayed to the user and the purchase process proceeds.

[0529] Step 12:

[0530] User: Uses the provided purchase link to purchase furniture through a shopping site or payment platform.

[0531] Server: Follow up on the purchase by sending the user a confirmation of purchase and tracking information.

[0532] Input: Purchase completion information

[0533] Output: Purchase confirmation notice, tracking information

[0534] Specific Operation: After the purchase is completed, the server generates a purchase confirmation notice and sends it to the user's mailbox.

[0535] (Application example 2)

[0536] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0537] Conventional interior coordination systems can suggest furniture based on room layout data and assist in purchasing at the optimal price, but they lack personalized suggestions that take into account the user's emotional state. Furthermore, users cannot actually try out the furniture they selected in the store in real time, making it difficult to visualize the furniture before purchasing. To solve these issues, there is a need for a system that can suggest interior coordination that takes into account the user's emotional state and that uses AR technology to simulate furniture in the real world.

[0538] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0539] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching for the listed furniture at the best price from multiple online shopping sites, means for calculating point benefits associated with purchases from the search results, means for providing information including the calculation results to the user, means for generating a purchase link to the product selected by the user, means for recognizing the user's emotional state, means for selecting an interior coordination plan based on the user's emotional state, and means for displaying a simulation of the real-world arrangement of furniture and interior items using AR technology. This enables personalized suggestions that take the user's emotional state into consideration, and allows for a real-time simulation of the arrangement of selected furniture in a store, making it easier to get an idea of ​​the appearance before purchasing.

[0540] "Room layout data" is spatial information including the structure, size, shape, furniture placement area, window and door positions, and the like of the room.

[0541] "Analysis" refers to the process of analyzing the received room layout data to identify the characteristics of the room.

[0542] An "interior coordination plan" is a plan for furniture placement and color schemes that is suggested based on the room layout data and emotional state.

[0543] "Emotion recognition means" is a function that analyzes the user's emotional state using facial recognition data, etc.

[0544] The "means for listing furniture" is a function for listing necessary furniture items based on the generated interior coordination plan.

[0545] The "means for searching for products at the best price" is a function for searching for the best price for listed furniture items from multiple shopping sites on the Internet.

[0546] The "means for calculating point benefits" is a function for calculating point benefits associated with purchases on a shopping site or payment platform.

[0547] The "means for generating a purchase link" is a function that generates a purchase link to a product selected by the user.

[0548] The "means for selecting an interior coordination plan based on an emotional state" is a function for selecting an optimal interior coordination plan based on emotional data acquired by the emotion recognition means.

[0549] "Means for displaying a simulation of furniture placement using AR technology" refers to a function that uses augmented reality technology to display a simulation of the listed furniture items being placed in the user's room.

[0550] This invention is a system that proposes optimal interior coordination to users based on room layout data and allows them to purchase that furniture at the optimal price. Furthermore, by combining it with an emotion engine, it is possible to recognize the user's emotional state and make suggestions based on that. To implement this invention, the following system is required.

[0551] System Configuration

[0552] The system mainly functions between a server, a terminal, and a user. The user inputs the room layout data using the terminal, and the system proposes an interior coordination plan based on the data and emotional data, and provides optimal purchasing options.

[0553] Hardware and Software Use

[0554] Hardware:

[0555] Facial Recognition Camera

[0556] Smartphone

[0557] software:

[0558] OpenAI® API: Used to generate emotion-based furniture recommendations.

[0559] EmotionRecognizer: A library for emotion analysis based on facial recognition data.

[0560] ARRenderer: A renderer that displays furniture simulations using AR technology.

[0561] OpenCV: A library used to capture and display camera footage.

[0562] Furniture API: An API for retrieving furniture data and comparing prices.

[0563] Data processing and data calculation

[0564] 1. The user takes a floor plan and photos of the room and uploads them to the device.

[0565] The device receives this floor plan data and sends it to a server, which temporarily stores it and analyzes it using AI algorithms. The analysis identifies spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[0566] 2. Based on the analysis results, the server generates multiple interior coordination plans, taking into consideration the user's style, color scheme, furniture placement, and other factors.

[0567] 3. The user inputs emotional data into the device. This emotional data can be obtained by facial recognition using a webcam, text input, or voice input. Once the device receives the emotional data, it is sent to the server. The server uses an emotion engine to analyze the emotional data and identify the user's emotional state.

[0568] 4. The server selects the most appropriate interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, it will suggest furniture with calming colors and soft textures.

[0569] 5. The server generates a list of necessary furniture items based on each interior coordination plan. It then uses APIs and scraping technology to search for products at the best prices across multiple shopping sites. It retrieves and lists the lowest price information (price, store, rating, availability, etc.) found for each furniture item. It also calculates the points rewards for each item.

[0570] 6. The server sends information including the calculation results to the terminal, which displays it to the user through an interface. The user reviews the proposed interior coordination plan and furniture list and selects the one they like. The terminal resends the user's selection to the server. The server generates a purchase link for the selected furniture item and sends it to the terminal. The terminal displays the purchase link to the user and supports the purchase process.

[0571] Usage example

[0572] Users use their smartphones in physical stores to upload their room layout data. The server analyzes this data and makes suggestions. The user then uses a camera in the store to provide emotional data, allowing the server to understand the user's desire for relaxation and suggest relaxing interior designs. These interior design suggestions are simulated in real time on the smartphone using AR technology, allowing the user to view them on the spot. Next, furniture with the best price and point benefits is listed, and a purchase link is generated for the furniture selected by the user, allowing for easy purchase procedures.

[0573] Example prompts for generative AI models

[0574] Room data: {"width": 5.0, "length": 6.0, "height": 2.5, "objects": []}

[0575] Emotion: Relaxed

[0576] Suggest furniture and layout:

[0577] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0578] Step 1:

[0579] Users take floor plans and photos of their rooms and upload them to their devices.

[0580] Input: Room floor plan and photos

[0581] Output: Floor plan data saved on the device

[0582] Specific operation: Launch the camera app on your smartphone, take a photo of the floor plan or room, and upload it to the app. The uploaded data is temporarily stored on the device.

[0583] Step 2:

[0584] The terminal receives the room layout data uploaded by the user and transmits it to the server via the interface.

[0585] Input: Floor plan data saved on the device

[0586] Output: Floor plan data sent to the server

[0587] Specific operation: The application on the device sends an HTTP request to the server for the saved floor plan data. The floor plan data is attached to the request.

[0588] Step 3:

[0589] The server temporarily stores the received floor plan data and prepares it for analysis.

[0590] Input: Submitted floor plan data

[0591] Output: Floor plan data saved on the server

[0592] Specific operation: The server receives an HTTP request and temporarily stores the floor plan data in a database. After storing it, it prepares the data so that the analysis program can call it.

[0593] Step 4:

[0594] The server analyzes the stored floor plan data using AI algorithms to identify spatial characteristics such as room size, shape, available furniture placement areas, and the location of windows and doors.

[0595] Input: Floor plan data

[0596] Output: Analysis results identifying room characteristics

[0597] How it works: An AI program on the server analyzes the floor plan data and extracts various room characteristics, including the room's dimensions, wall position, and window and door placement.

[0598] Step 5:

[0599] The server generates a plurality of interior coordination plans based on the analysis results.

[0600] Input: Analysis results

[0601] Output: Interior coordination plan

[0602] How it works: The server's coordinate generator generates different designs and furniture layout plans based on the analysis results, including suggestions based on different styles and themes.

[0603] Step 6:

[0604] The user can input emotion data into the terminal by using face recognition via a webcam, text input, or voice input.

[0605] Input: Emotion data (facial recognition data, text, voice)

[0606] Output: Emotion data stored on the device

[0607] What it does: Users can use their smartphone camera to take a picture of their face so the application can analyze emotional data, or they can use text or voice input methods.

[0608] Step 7:

[0609] The terminal receives the user's emotion data and transmits it to the server.

[0610] Input: Emotion data stored on the device

[0611] Output: Emotion data sent to the server

[0612] Specific operation: The device application sends an HTTP request to the server with emotion data.

[0613] Step 8:

[0614] The server uses an emotion engine to analyze the received emotion data and identify the user's emotional state.

[0615] Input: Emotion data

[0616] Output: Emotional state analysis result

[0617] Specific operation: The emotion engine on the server analyzes the emotion data and identifies the user's emotional state (e.g., relaxed, excited, stressed, etc.).

[0618] Step 9:

[0619] The server selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine.

[0620] Input: Emotional state analysis results, interior coordination plan

[0621] Output: Optimal interior coordination plan based on emotional state

[0622] Specific operation: The server's selection program selects the interior coordination plan that best suits the user's emotional state.

[0623] Step 10:

[0624] The server generates a list of necessary furniture items based on each interior coordination plan.

[0625] Input: Interior coordination plan

[0626] Output: A list of furniture items

[0627] How it works: The server's database and algorithms create a list of the furniture items required based on the selected plan.

[0628] Step 11:

[0629] The server then begins searching for the listed furniture items using APIs and scraping technology to find the best prices on multiple online shopping sites.

[0630] Input: A list of furniture items

[0631] Output: Product information at the best price

[0632] How it works: The server's search algorithm uses each shopping site's API and scraping technology to obtain price information for the listed furniture items.

[0633] Step 12:

[0634] The server retrieves and lists the cheapest product information (price, store, rating, availability, etc.) found for each furniture item.

[0635] Input: Search results

[0636] Output: Information about the cheapest product

[0637] Specific operation: Based on the obtained price information, the server lists the cheapest products and saves them in the database.

[0638] Step 13:

[0639] Based on the search results, the server calculates the point rewards (such as points redemption on shopping sites or payment platforms) for each product.

[0640] Input: Information about the cheapest product

[0641] Output: Point reward information

[0642] Specific operation: The server calculates the point reward information for each product based on the point redemption rules of the shopping site or payment platform.

[0643] Step 14:

[0644] The server transmits the calculated reward information and the search results to the terminal.

[0645] Input: Point reward information, lowest price product information

[0646] Output: Interior coordination plan, furniture list, and point reward information sent to the terminal

[0647] Specific operation: The server sends the calculation results and point reward information together to the terminal so that the user can check them.

[0648] Step 15:

[0649] The terminal displays the interior coordination plan, furniture list, and point benefit information received from the server to the user through an interface.

[0650] Input: Interior coordination plan, furniture list, point reward information

[0651] Output: Information displayed in the interface

[0652] Specific operation: The terminal application displays the information received from the server in the user interface and allows the user to make a selection.

[0653] Step 16:

[0654] The user checks the proposed interior coordination plan and furniture list and selects the one they like.

[0655] Input: User's choice

[0656] Output: User selection information

[0657] Specific operation: The user operates the terminal interface to select the desired item from the proposed coordination plan and furniture list.

[0658] Step 17:

[0659] The terminal accepts the user's selection and retransmits the information to the server.

[0660] Input: User selection information

[0661] Output: Selections sent to the server

[0662] Specific operation: The terminal makes an HTTP request again to send the user's selection information to the server.

[0663] Step 18:

[0664] The server generates a purchase link for the furniture item selected by the user and sends it to the terminal.

[0665] Input: User selection information

[0666] Output: Purchase link

[0667] Specific operation: The server generates a purchase link for the specified furniture item and sends it to the device.

[0668] Step 19:

[0669] The terminal displays a purchase link to the user and assists in the purchase procedure.

[0670] Enter: Purchase Link

[0671] Output: Display link to user

[0672] What happens: The device displays a purchase link in the user interface and helps the user click the link to actually complete the purchase.

[0673] Step 20:

[0674] Users use the provided purchase link to purchase the furniture through a shopping site or payment platform.

[0675] Enter: Purchase Link

[0676] Output: Purchase complete

[0677] What happens: The user clicks on the purchase link, is taken to the shopping site, and completes the purchase.

[0678] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0679] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0680] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0681] [Second embodiment]

[0682] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0683] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0684] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0685] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0686] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0687] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0688] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0689] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0690] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0691] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0692] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0693] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0694] This invention relates to a system that proposes optimal interior coordination to users based on room layout data and also allows them to purchase the furniture at the optimal price. The specific operations performed in the processing of each step are described below.

[0695] System Configuration

[0696] The system mainly works between a server, a terminal, and a user. The user inputs the layout data of the room using the terminal, and the server uses that data to suggest interior coordination and provide optimal purchasing options.

[0697] Program processing

[0698] 1. Enter and submit room layout data

[0699] User: Takes a floor plan and photographs of the room, and uploads them to the system via a device.

[0700] Terminal: Sends the uploaded data to the server.

[0701] Server: Temporarily stores received data and prepares it for analysis.

[0702] 2. Analysis of room layout data

[0703] Server: Analyzes the received floor plan data using an AI algorithm to identify spatial characteristics such as room size, shape, available furniture placement areas, and the location of windows and doors.

[0704] Example: If a floor plan is divided into a living room, bedroom, and kitchen, the system automatically recognizes the area and shape of each room, as well as the space available for furniture placement.

[0705] 3. Creating an interior coordination plan

[0706] Server: Based on the analysis results, it generates multiple interior coordination plans according to the purpose of use of the space and the user's preferences.

[0707] Example: For a living room, suggestions include a Scandinavian-style furniture arrangement plan and a modern furniture arrangement plan.

[0708] 4. List your furniture items

[0709] Server: Generates a list of required furniture items based on the proposed interior coordination plan.

[0710] Example: Select key items for each category, such as sofas, dining tables, and beds.

[0711] 5. Find the best priced furniture

[0712] Server: Searches multiple online shopping sites for the best price for the listed furniture items.

[0713] Example: Search for "L-shaped sofa" on multiple shopping sites and determine the best product based on price, reviews, delivery terms, etc.

[0714] 6. Calculation of points rewards

[0715] Server: Calculates the points reward for each product based on the search results.

[0716] Example: Calculate the reward points when purchasing using a combination of Yahoo! Shopping and PayPay.

[0717] 7. Providing Information to the User Interface

[0718] Server: Sends calculated reward information and search results to the device.

[0719] Terminal: Displays proposed interior plans and product lists to the user through a user interface.

[0720] 8. Generate Purchase Link

[0721] User: Select the best from the proposed interior plans and product list.

[0722] Server: Generates a purchase link for the product selected by the user and sends it to the device.

[0723] Device: Show the purchase link to the user and assist them in the purchase process.

[0724] Specific examples

[0725] A user wants to redecorate their living room, so they upload a floor plan. The server analyzes the living room, recognizes its size and shape, and suggests furniture placement. It then searches the internet for the best prices for each furniture item and calculates the combination that maximizes point rewards. Finally, it generates a purchase link and presents it to the user, allowing them to easily purchase the furniture.

[0726] This system allows users to effortlessly coordinate their interiors and purchase furniture economically all in one place.

[0727] The processing flow will be explained below.

[0728] Step 1:

[0729] User: Take a floor plan or photo of the room and upload it to the device.

[0730] Step 2:

[0731] Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[0732] Step 3:

[0733] Server: Temporarily stores the received floor plan data and prepares it for data analysis.

[0734] Step 4:

[0735] Server: Analyzes stored floor plan data using AI algorithms to identify spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[0736] Step 5:

[0737] Server: Generates multiple interior coordination plans based on the analysis results, proposing plans that take into account style, color scheme, furniture placement, etc.

[0738] Step 6:

[0739] Server: Generates a list of required furniture items based on each interior coordination plan.

[0740] Step 7:

[0741] Server: Initiates a search for the listed furniture items using APIs and scraping technology to find the best prices from multiple shopping sites.

[0742] Step 8:

[0743] Server: Retrieves and lists the cheapest price information (price, retailer, ratings, availability, etc.) found for each furniture item.

[0744] Step 9:

[0745] Server: Based on the search results, calculate the points rewards for each product (such as Yahoo! Shopping or PayPay points).

[0746] Step 10:

[0747] Server: Sends the interior coordination plan and furniture list including the calculation results to the terminal.

[0748] Step 11:

[0749] Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through an interface.

[0750] Step 12:

[0751] User: Review the proposed interior coordination plan and furniture list and select the one they like.

[0752] Step 13:

[0753] Terminal: Accepts the user's selection and resends the information to the server.

[0754] Step 14:

[0755] Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[0756] Step 15:

[0757] Device: Show the purchase link to the user and assist them in the purchase process.

[0758] Step 16:

[0759] User: Use the provided purchase link to purchase furniture through a shopping site or PayPay.

[0760] Step 17:

[0761] Server: Follow up with the user after the purchase by sending them a confirmation of the purchase and tracking information.

[0762] Example 1

[0763] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0764] Conventional interior coordination systems lacked the functionality to suggest optimal furniture placement to users. Furthermore, providing information on how to purchase furniture at the best price and calculating point rewards associated with purchases were often done manually, which was time-consuming for users. Furthermore, the systems lacked the means to safely handle users' personal information, leaving them at risk of data theft and unauthorized access.

[0765] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0766] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching multiple online shopping sites for products at the best price for the listed furniture, means for calculating points related to purchases from the search results, means for providing the user with information including the calculation results, means for generating a purchase link to the product selected by the user, and means for encrypting and transmitting data to protect the user's privacy. This allows the user to effortlessly coordinate an optimal interior and purchase furniture economically all in one place.

[0767] "Room layout data" refers to data provided by users that includes information such as the dimensions and shape of the room, the area in which furniture can be placed, and the positions of windows and doors.

[0768] The "receiving means" is a system or device for receiving room layout data from a user.

[0769] "Means for analysis" refers to a system or device that analyzes the received room layout data using an AI algorithm or the like to identify the structure and characteristics of the room.

[0770] A "means for identifying features" is a system or device that has the ability to extract important spatial information from the analysis results, such as the size and shape of the room, the area where furniture can be placed, and the location of windows and doors.

[0771] An "interior coordination plan" is data that shows furniture placement and design plans proposed based on the user's preferences and room characteristics.

[0772] The "generating means" is a system or device for creating a plurality of interior coordination plans based on the analysis results.

[0773] The "listing means" is a system or device having a function for creating a list of necessary furniture and decorative items based on the generated interior coordination plan.

[0774] The "means for searching for products at the best price" is a system or device that has the function of searching for the most favorable priced product for the listed furniture from multiple online shopping sites.

[0775] The "means for calculating point benefits" is a system or device that has the function of calculating the total points redeemed based on the point redemption information provided by multiple shopping sites on the Internet based on search results.

[0776] The "means for providing information including calculation results to users" refers to a system or device for notifying users of search results including point redemption information and visually displaying them.

[0777] The "means for generating a purchase link" refers to a system or device for creating a link that allows a user to directly purchase the product selected by the user on an online shopping site.

[0778] "Means for encryption and transmission" refers to a system or device that encrypts data such as floor plan data and user information and communicates them securely in order to protect user privacy.

[0779] This invention relates to a system that proposes optimal interior coordination to users based on room layout data and allows them to purchase the furniture at the optimal price. This system functions in cooperation with a server, terminals, and users.

[0780] System Configuration

[0781] The user inputs the layout data of the room using a terminal, and the server uses that data to propose an interior coordination plan and also provides optimal purchasing options. In terms of specific hardware, the user's terminal can be a smartphone, tablet, or PC, while the server uses a high-performance computer system.

[0782] Entering and Submitting Data

[0783] User

[0784] Users take photos of the room floor plan and other images using the camera on their smartphone or PC, and upload the data to the system using the following prompts:

[0785] "I uploaded the living room floor plan."

[0786] "I sent you the bedroom floor plan."

[0787] Terminal

[0788] The terminal temporarily stores the data entered by the user and transmits it to the server using a secure communication protocol (HTTPS).

[0789] server

[0790] The server temporarily stores the received data and checks the data for consistency and format. If any errors are found, the user is notified and asked to re-upload.

[0791] Data analysis

[0792] server

[0793] The server uses AI algorithms (e.g., Convolutional Neural Networks) to analyze the data it receives, identifying important spatial features such as the room's size, shape, available furniture area, and the location of windows and doors.

[0794] As a specific example, if a floor plan is divided into a living room, bedroom, and kitchen, the area and shape of each room, as well as the space available for furniture placement, are automatically recognized using libraries such as HTML5 Canvas and OpenCV.

[0795] Coordination plan generation

[0796] server

[0797] The server generates multiple interior coordination plans based on the above analysis results, tailored to the user's preferences and lifestyle. Using a specific generation AI model, the server generates plans using the following prompts:

[0798] "Please suggest a Scandinavian-style interior design for the living room."

[0799] "Generate a modern bedroom furniture arrangement plan."

[0800] The generated plans are offered for selection by the user.

[0801] Listing furniture items

[0802] server

[0803] Based on the plan generated, create a list of the furniture items you'll need, including items in each major category, such as a sofa, dining table, and bed.

[0804] Find the best price

[0805] server

[0806] The server searches multiple online shopping sites in real time for the best price for the listed furniture, taking into account not only price but also ratings, delivery terms, and other factors.

[0807] Points Reward Calculation

[0808] server

[0809] Based on the search results, the points rewards for each product are calculated. For example, the total points are calculated based on the point reward rates on Yahoo! Shopping and other online platforms.

[0810] Providing information

[0811] server

[0812] Information including the calculation results is sent to the terminal, which then visually displays it so that the user can easily understand it.

[0813] Generate purchase links

[0814] User

[0815] The user selects the most suitable item from the proposed interior plans and product list.

[0816] server

[0817] The server generates a purchase link for the product selected by the user and sends it to the terminal, which displays the link to the user and assists in the purchase process.

[0818] Data encryption

[0819] Terminals and Servers

[0820] To protect user privacy, all data transmitted is encrypted using Transport Layer Security (TLS), which protects user information from data theft and unauthorized access.

[0821] As described above, this system minimizes the user's effort and enables optimal interior coordination and economical furniture purchases.

[0822] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0823] Step 1:

[0824] input

[0825] User: Take a floor plan or photo of a room.

[0826] Users: Upload data via a dedicated application or web portal.

[0827] output

[0828] Device: Temporarily stores floor plans and photo data.

[0829] Terminal: Sends data to the server using a secure communication protocol (HTTPS).

[0830] Specific actions

[0831] Users take a picture of the room floor plan using the camera on their smartphone or PC.

[0832] The captured data is saved in the application and sent to the server.

[0833] Step 2:

[0834] input

[0835] Terminal: Receives floor plan data from the user.

[0836] Server: Receives floor plan data sent from the device.

[0837] output

[0838] Server: Temporarily stores the received data and checks its structural integrity.

[0839] Server: If the data is in the correct format, prepare it for analysis. If it is not, send an error notification to the user.

[0840] Specific actions

[0841] The server checks the integrity of the data and notifies the user if there are any discrepancies.

[0842] If the data format is correct, proceed to the next analysis step.

[0843] Step 3:

[0844] input

[0845] Server: Received floor plan data

[0846] output

[0847] Server: Analyzes floor plan data and identifies room size, shape, available furniture placement area, and window and door locations.

[0848] Specific actions

[0849] The server analyzes the floor plan data using an AI algorithm (e.g., Convolutional Neural Network).

[0850] Identify room boundaries, dimensions, and functional areas and store them in a database.

[0851] Step 4:

[0852] input

[0853] Server: Analysis results (room size, shape, furniture placement area, etc.)

[0854] output

[0855] Server: Multiple interior coordination plans based on user preferences

[0856] Specific actions

[0857] The server uses a generative AI model to generate a coordination plan based on the prompt sentence.

[0858] Example: "Generate a Scandinavian furniture arrangement plan for my living room."

[0859] Organize the proposed plans and select the most suitable one.

[0860] Step 5:

[0861] input

[0862] Server: Generated interior coordination plan

[0863] output

[0864] Server: List of required furniture items

[0865] Specific actions

[0866] The server lists the necessary furniture items based on the coordination plan.

[0867] Select key items for each category, such as sofas and dining tables.

[0868] Step 6:

[0869] input

[0870] Server: List of furniture items

[0871] output

[0872] Server: A list of products with the best prices from multiple shopping sites on the Internet

[0873] Specific actions

[0874] The server crawls multiple shopping sites and compares prices.

[0875] Example: Search for "L-shaped sofa" across multiple sites to identify the best products.

[0876] Product ratings and delivery terms are also taken into consideration.

[0877] Step 7:

[0878] input

[0879] Server: Price comparison results

[0880] output

[0881] Server: Calculation result of point benefits for each product

[0882] Specific actions

[0883] The server calculates the point rewards for each product based on the point redemption rate.

[0884] Example: Calculating Yahoo! Shopping reward points.

[0885] Step 8:

[0886] input

[0887] Server: Calculated point reward information

[0888] Terminal: Information received from the server

[0889] output

[0890] Terminal: Displays interior plans and product lists in a user-friendly format

[0891] Specific actions

[0892] The terminal retrieves the calculated reward information and displays it in a graphical interface.

[0893] Product images, price information, point redemption information, and more can be viewed at a glance.

[0894] Step 9:

[0895] input

[0896] User: Select the furniture item they want from the list of suggested items

[0897] output

[0898] Server: Purchase link for selected product

[0899] Specific actions

[0900] The server generates a purchase link for the furniture item selected by the user.

[0901] Send a purchase link to the device to allow users to easily make a purchase.

[0902] By following the above steps, users can efficiently and easily coordinate their interiors optimally and purchase furniture economically.

[0903] (Application example 1)

[0904] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0905] Currently, when coordinating interiors, it is difficult to select and arrange the optimal furniture to fit the floor plan. Furthermore, there is a lack of ways to purchase furniture at the best price or to virtually check what the interior will look like after purchase. This requires users to spend a lot of time and effort, making it difficult to achieve effective interior coordination.

[0906] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0907] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching multiple online shopping sites for products at the best prices for the listed furniture, means for calculating point benefits associated with purchases from the search results, means for providing the user with information including the calculation results, means for generating purchase links to products selected by the user, means for the user to 3D scan the floor plan data, and means for confirming the interior coordination plan in a virtual space using the 3D scanned data. This allows the user to efficiently and economically realize optimal interior coordination and virtually confirm it.

[0908] "Room layout data" is information that indicates the layout, size, and shape of a physical space such as a user's residence or office.

[0909] "Means for receiving" refers to devices or software that have the functionality to allow the system to acquire data sent from the user.

[0910] "Means for analysis" refers to algorithms or software that analyze the received data and identify the structure and characteristics of the space.

[0911] An "interior coordination plan" is a specific proposal for the layout and design of furniture and decorations to be placed in a room.

[0912] The "listing means" refers to a device or software that has the function of selecting the necessary furniture and products based on the interior coordination plan.

[0913] "Internet shopping site" refers to a website where users can search for and purchase products online.

[0914] A "means for searching for products at the best price" is an algorithm or software that searches multiple shopping sites for products that meet your criteria.

[0915] The "means for calculating point benefits" refers to a device or software that has the function of calculating points or discounts that can be obtained at the time of purchase.

[0916] "Means for providing information to users" refers to the interface or software for displaying calculation results and proposals to users.

[0917] The "means for generating a purchase link" refers to a device or software that has the function of creating a link for purchasing the product selected by the user.

[0918] "3D scanning means" refers to devices and software for digitizing physical spaces as three-dimensional data.

[0919] "Means for checking in a virtual space" refers to devices or software that have the function of allowing users to visually check interior coordination plans using three-dimensional models.

[0920] The present invention relates to a system that allows a user to propose optimal interior coordination based on room layout data, and detailed embodiments thereof will be described.

[0921] System Configuration

[0922] The system mainly functions between the server, the terminal, and the user. The system receives room layout data and 3D scan data from the user and proposes interior coordination in the virtual space. Specifically, it works as follows:

[0923] Hardware and Software

[0924] Hardware:

[0925] A smartphone or head-mounted display (e.g., Oculus Quest 2)

[0926] Server (Cloud server or on-premise server)

[0927] software:

[0928] Unity, Vuforia (AR technology), OpenCV

[0929] Python and Flask (server-side processing)

[0930] MongoDB (data storage)

[0931] Processing flow

[0932] Data reception and analysis

[0933] 1. User:

[0934] Use a smartphone or head-mounted display to 3D scan the layout of a room.

[0935] The acquired data is sent from the terminal to the server.

[0936] 2. Server:

[0937] The server analyzes the received 3D scan data using OpenCV to determine the size, shape, and area where furniture can be placed in the room.

[0938] The analysis results are saved in MongoDB.

[0939] Generate interior coordination plans

[0940] 3. Server:

[0941] Based on the analysis results, a Python script is used to generate multiple interior coordination plans, such as Scandinavian and modern styles.

[0942] Check in virtual space

[0943] 4. User:

[0944] Using Unity and Vuforia, participants can visually check 3D models of proposed interior coordination plans in a virtual space using a head-mounted display or smartphone.

[0945] Product listing and price search

[0946] 5. Server:

[0947] The system creates a list of furniture items based on the generated interior coordination plan and uses a Python script to search multiple online shopping sites for the best prices.

[0948] Calculate points rewards for each product based on search results.

[0949] Purchase links and information

[0950] 6. Server:

[0951] The search results and calculated point reward information are sent to the terminal and displayed to the user.

[0952] A purchase link is generated for the product selected by the user and sent to the terminal.

[0953] 7. Users:

[0954] Check the proposed interior plans and product list and select the most suitable products.

[0955] Easily purchase products through our purchase links.

[0956] Prompt Sentence Examples

[0957] The invention provides a virtual interior coordination assistant application that allows users to send scanned data of a room to a server and easily coordinate the optimal interior and compare prices.

[0958] As a concrete example, a user considers redecorating their bedroom and scans the room using a head-mounted display. The system analyzes the size and shape of the room and proposes interior plans in both Scandinavian and modern styles. The user can then view the virtual 3D models and purchase furniture that suits their tastes at the best price.

[0959] This system allows users to efficiently coordinate their interiors optimally and purchase furniture at economical prices.

[0960] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0961] Step 1:

[0962] Users use a smartphone or head-mounted display to 3D scan the layout of a room, and the scanned data is then sent from the device to a server.

[0963] Input: 3D scan of a room.

[0964] Processing: The device receives the 3D scan data taken by the user, converts it into the appropriate format, and sends it to the server.

[0965] Output: 3D scan data sent to the server.

[0966] Step 2:

[0967] The server analyzes the received 3D scan data to determine the size, shape, and available furniture area of ​​the room using OpenCV.

[0968] Input: 3D scan data.

[0969] Processing: Using OpenCV, the server analyzes the 3D scan data and extracts spatial features (size, shape, furniture placement area), applying image processing techniques such as edge detection and shape analysis.

[0970] Output: Data about the room size, shape, and available furniture area.

[0971] Step 3:

[0972] The server generates multiple interior coordination plans based on the analysis results, using a Python script.

[0973] Input: Data about the room size, shape, and available furniture area.

[0974] Processing: Using a Python script, multiple interior coordination plans (e.g., Scandinavian style, modern style) are generated based on the analysis results. Various plans are created by referencing past design data and user preferences.

[0975] Output: Interior coordination plan data.

[0976] Step 4:

[0977] Using Unity and Vuforia, users can visually view 3D models of proposed interior coordination plans in a virtual space using a head-mounted display or smartphone.

[0978] Input: Interior coordination plan data.

[0979] Processing: Using Unity and Vuforia, the interior coordination plan is created as a 3D model and displayed in a user interface, allowing users to visually inspect the model and perform operations such as rotating and scaling.

[0980] Output: A virtual interior coordination that can be visually confirmed by the user.

[0981] Step 5:

[0982] The server lists furniture items based on the interior coordination plan and uses a Python script to search multiple shopping sites on the Internet for the best priced items.

[0983] Input: Interior coordination plan data.

[0984] Processing: The server crawls the web for the listed furniture items, collects information from multiple shopping sites, and compares and selects the best options based on price, reviews, delivery terms, etc.

[0985] Output: A list of the best purchasing options for each furniture item.

[0986] Step 6:

[0987] The server calculates the points reward for each product based on the search results.

[0988] Enter: the best buying options list.

[0989] Processing: The server obtains the point reward information of each shopping site and calculates the number of points that can be earned at the time of purchase. This is done to calculate the economic efficiency of the purchase.

[0990] Output: A list of optimal purchase options with point rewards.

[0991] Step 7:

[0992] The server generates the calculation results and a link to purchase the furniture, and sends them to the device. The user can then use their smartphone or head-mounted display to check the interior plan and product list and select the most suitable product. The product can then be easily purchased using the purchase link.

[0993] Enter: a list of the best purchasing options, including points rewards.

[0994] Processing: Organize the calculated reward information and product list, generate data to display in the user interface, generate a purchase link for the product selected by the user, and send it to the device.

[0995] Output: Interior plans and product list with purchase links.

[0996] This series of processes allows users to efficiently and economically create optimal interior coordination and easily purchase furniture while checking it virtually.

[0997] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0998] This invention combines an emotion engine with a system that proposes optimal interior coordination to users based on room layout data and then allows them to purchase that furniture at the optimal price. The emotion engine recognizes the user's emotional state and makes suggestions based on that.

[0999] System Configuration

[1000] The system mainly works between the server, the terminal, and the user. The user inputs the room layout data using the terminal, and the server proposes interior coordination based on that data and the user's emotional data, and provides optimal purchasing options.

[1001] Program processing

[1002] Enter and submit room layout data

[1003] 1. User: Take a floor plan or photo of the room and upload it to the device.

[1004] 2. Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[1005] 3. Server: Temporarily stores the received data and prepares it for analysis.

[1006] Analysis of room layout data

[1007] 4. Server: The stored floor plan data is analyzed using AI algorithms to identify spatial characteristics such as room size, shape, furniture placement area, and window and door locations.

[1008] Generate interior coordination plans

[1009] 5. Server: Based on the analysis results, it generates multiple interior coordination plans, taking into account style, color scheme, furniture placement, etc.

[1010] Emotion data input and analysis

[1011] 6. User: Emotional data can be input to the device using facial recognition via a webcam, text input, or voice input.

[1012] 7. Terminal: Receives the user's emotion data and sends it to the server.

[1013] 8. Server: Analyzes the received emotion data using the emotion engine and identifies the user's emotional state.

[1014] Generate personalized proposals

[1015] 9. Server: Selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, it will suggest furniture with calming colors and soft textures.

[1016] Listing furniture items

[1017] 10. Server: Generates a list of required furniture items based on each interior coordination plan.

[1018] Find the best priced furniture

[1019] 11. Server: Initiates a search for the listed furniture items using APIs and scraping techniques to find the best prices from multiple shopping sites.

[1020] 12. Server: Retrieves and lists the cheapest price information (price, retailer, rating, availability, etc.) found for each furniture item.

[1021] Points Reward Calculation

[1022] 13. Server: Based on the search results, calculates the points rewards for each product (such as points redemption on shopping sites or payment platforms).

[1023] Providing information to the user interface

[1024] 14. Server: Sends the calculated reward information and search results to the device.

[1025] 15. Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through the interface.

[1026] Generate purchase links

[1027] 16. User: Review the proposed interior coordination plan and furniture list and select the one they like best.

[1028] 17. Terminal: Accepts the user's selection and resends the information to the server.

[1029] 18. Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[1030] 19. Device: Display the purchase link to the user and assist with the purchase process.

[1031] Purchasing and Follow-up

[1032] 20. User: Uses the provided purchase link to purchase furniture through a shopping site or payment platform.

[1033] 21. Server: Follow up on the purchase by sending the user a confirmation of the purchase and tracking information.

[1034] Specific examples

[1035] A user wants to redecorate their living room and uploads a floor plan. The server analyzes the living room, recognizes its size and shape, and suggests furniture placement. The user then provides emotional data via webcam, revealing that they are looking for a relaxing space. Based on this information, the server suggests furniture with calming color schemes and interior design that creates a relaxing effect. The server searches multiple shopping sites for the best prices and point rewards for the furniture and provides them to the user. Once the user selects the furniture, a purchase link is generated to help them easily complete the purchase process.

[1036] This system allows users to coordinate their interiors to best suit their emotional state and makes hassle-free and economical furniture purchases all in one go.

[1037] The processing flow will be explained below.

[1038] Step 1:

[1039] User: Take a floor plan or photo of the room and upload it to the device.

[1040] Step 2:

[1041] Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[1042] Step 3:

[1043] Server: Temporarily stores the received floor plan data and prepares it for data analysis.

[1044] Step 4:

[1045] Server: Analyzes stored floor plan data using AI algorithms to identify spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[1046] Step 5:

[1047] Server: Generates multiple interior coordination plans based on the analysis results, proposing plans that take into account style, color scheme, furniture placement, etc.

[1048] Step 6:

[1049] User: Enters emotion data into the device using face recognition via a webcam, text input, or voice input.

[1050] Step 7:

[1051] Terminal: Receives the user's emotion data and sends the emotion data to the server.

[1052] Step 8:

[1053] Server: Analyzes the received emotional data using the emotion engine and identifies the user's emotional state. If the user wants to relax, it will be labeled as "Relaxed," and if the user wants to be energized, it will be labeled as "Energy."

[1054] Step 9:

[1055] Server: Selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, the server will suggest furniture with calming colors and soft textures.

[1056] Step 10:

[1057] Server: Generates a list of required furniture items based on each interior coordination plan.

[1058] Step 11:

[1059] Server: Initiates a search for the listed furniture items using APIs and scraping technology to find the best prices from multiple shopping sites.

[1060] Step 12:

[1061] Server: Retrieves and lists the cheapest price information (price, retailer, ratings, availability, etc.) found for each furniture item.

[1062] Step 13:

[1063] Server: Based on the search results, calculates the reward points for each product (such as points redemption from shopping sites or payment platforms).

[1064] Step 14:

[1065] Server: Sends the interior coordination plan and furniture list including the calculation results to the terminal.

[1066] Step 15:

[1067] Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through an interface.

[1068] Step 16:

[1069] User: Review the proposed interior coordination plan and furniture list and select the one they like.

[1070] Step 17:

[1071] Terminal: Accepts the user's selection and resends the information to the server.

[1072] Step 18:

[1073] Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[1074] Step 19:

[1075] Device: Show the purchase link to the user and assist them in the purchase process.

[1076] Step 20:

[1077] User: Purchases furniture through a shopping site or payment platform using the provided purchase link.

[1078] Step 21:

[1079] Server: Follow up with the user after the purchase by sending them a confirmation of the purchase and tracking information.

[1080] Example 2

[1081] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1082] Conventional interior coordination suggestion systems have difficulty making personalized suggestions that take into account the user's emotional state. Furthermore, searching for furniture prices and presenting point benefits is time-consuming, and the system provides insufficient support for users to make optimal choices. This makes it difficult to achieve interior coordination that satisfies users and to efficiently purchase furniture.

[1083] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for acquiring and analyzing the user's emotional state, means for selecting an appropriate interior coordination plan based on the acquired emotional state, means for listing furniture based on the generated interior coordination plan, means for searching multiple shopping sites on the Internet for products at the best prices for the listed furniture, means for calculating point benefits related to purchases from the search results, means for providing the user with information including the calculation results, and means for generating purchase links to products selected by the user. This allows the server to suggest optimal interior coordination based on the user's emotional state, enabling efficient and economical furniture purchases.

[1084] "Room layout data" is numerical and graphical information that indicates the shape and dimensions of a room, the area in which furniture can be placed, the positions of windows and doors, and so on.

[1085] An "interior coordination plan" is a proposal for the interior design of a room, including the style, color scheme, furniture placement, and decorative items.

[1086] "Emotional state" is information that indicates the user's mood and state of mind, and is primarily obtained through facial recognition, voice analysis, text input, etc.

[1087] A "furniture list" is a list of necessary furniture items based on a specific interior coordination plan, including detailed information about each item.

[1088] The "best price" is the lowest price obtained from multiple shopping sites on the Internet for the furniture item specified by the user.

[1089] "Point benefits" are points or discount information provided by shopping sites or payment platforms that are given back to users when they make a purchase.

[1090] A "purchase link" is a direct URL or hyperlink to an online shopping site that allows a user to purchase a specified furniture item.

[1091] This invention is a system that proposes optimal interior coordination to users based on room layout data. This system makes personalized suggestions based on the user's emotional state, and also allows the user to purchase furniture at the optimal price based on those suggestions.

[1092] System Configuration

[1093] This system mainly functions between a server, a terminal, and a user. The user inputs room layout data using the terminal, and the server proposes interior coordination based on that data and the user's emotional data, and provides optimal purchasing options.

[1094] Hardware and software used

[1095] Devices: Smartphone, tablet, PC (with webcam)

[1096] Server: Cloud server (e.g. Amazon Web Services)

[1097] AI algorithm: TensorFlow (for image analysis)

[1098] Emotion analysis engine: Emotion API (face recognition, voice analysis)

[1099] Database: SQL database (for managing customer information and furniture lists)

[1100] API: Web scraping tool (for product price search)

[1101] Program processing

[1102] Enter and submit room layout data

[1103] The user uses the device to take floor plans and photos of the room and uploads them to the device through a dedicated application. The device receives the data uploaded by the user and sends it to the server.

[1104] Analysis of room layout data

[1105] The server analyzes the received floor plan data using an AI algorithm (e.g., TensorFlow) to identify spatial characteristics such as room size, shape, and available furniture placement areas, as well as the location of windows and doors.

[1106] Generate interior coordination plans

[1107] The server generates multiple interior coordination plans based on the analysis results, including different styles, color schemes, and furniture placements.

[1108] Emotion data input and analysis

[1109] Users can input emotion data into their devices using facial recognition via a webcam, text input, or voice input. The device receives the user's emotion data and sends it to a server. The server then uses an emotion analysis engine (e.g., Emotion API) to identify the user's emotional state.

[1110] Generate personalized proposals

[1111] The server selects the most appropriate interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking to relax, it will suggest furniture with a calm color scheme and soft textures.

[1112] Listing furniture items

[1113] The server generates a list of necessary furniture items based on the selected interior plan, including detailed information about each piece (size, design, material, etc.).

[1114] Find the best priced furniture

[1115] The server searches for the best priced products for the listed furniture items from multiple shopping sites using APIs and scraping technology, and obtains the best prices and ratings for the furniture based on the search results.

[1116] Points Reward Calculation

[1117] The server calculates the points rewards (points redeemed by shopping sites and payment platforms) for each product, allowing users to make the most economical overall choice.

[1118] Providing information to the user interface

[1119] The server transmits the calculated reward information and search results to the terminal, which displays the information to the user through an interface.

[1120] Generate purchase links

[1121] The user checks the presented interior coordination plan and furniture list and selects the furniture they want. The terminal retransmits the user's selection to the server. The server generates a purchase link for the furniture item selected by the user and transmits it again to the terminal. The terminal displays the purchase link to the user and supports the purchase procedure.

[1122] Purchasing and Follow-up

[1123] The user uses the provided purchase link to purchase the furniture through a shopping site or payment platform, and the server then sends the user a confirmation of the purchase and tracking information as follow-up after the purchase.

[1124] Specific operation example

[1125] For example, if a user is thinking about redecorating their living room, they first upload a floor plan. The server analyzes the floor plan and recognizes the size and shape of the living room. Next, the user provides emotional data via webcam, and it is determined that they are looking for a relaxing space. Based on this information, the server suggests calming color schemes and interior design that has a relaxing effect. It also searches multiple shopping sites for the best prices and point rewards for furniture and provides them to the user. The server then generates a purchase link for the furniture selected by the user, facilitating a simple purchasing process.

[1126] Prompt Sentence Examples

[1127] I'm thinking of redecorating my living room. Please suggest the best interior design based on this floor plan. I want to create a relaxing space.

[1128] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1129] Step 1:

[1130] User: Takes a photo or a floor plan of the room and uploads it to the device via a dedicated application. The request includes the floor plan and photo data. For example, a user takes a photo of the living room floor plan with their smartphone and presses the "upload" button.

[1131] Terminal: Receives floor plan data uploaded by the user, checks the data format (JPEG or PNG), and sends it to the server. Once the data format check is complete, it is sent to the server.

[1132] Specific operation: When the user presses the "Upload floor plan" button, the image data of the floor plan is saved on the device and transferred to the server.

[1133] Step 2:

[1134] Server: The received floor plan data is temporarily stored in cloud storage (e.g., Amazon S3), and then an AI algorithm (TensorFlow) is invoked to analyze it.

[1135] Input: Received image data of floor plans and photos

[1136] Output: Analytical data including room size, shape, available furniture area, and spatial characteristics such as window and door locations

[1137] Specific operation: The server passes the stored floor plan to an AI algorithm, which performs edge detection and object recognition. The analysis results are then stored in a database.

[1138] Step 3:

[1139] Server: Based on the analysis results, the server executes the interior coordination plan generation program. Here, it references several interior style templates from the database and selects a plan that suits the user's room.

[1140] Input: Room analysis data

[1141] Output: Multiple interior coordination plans (style, color scheme, furniture placement)

[1142] Specific operation: Based on the analysis data, multiple optimal templates are extracted and images and text information for presentations are generated.

[1143] Step 4:

[1144] User: Activate the device's webcam and use the facial recognition system to input emotion data, or choose to input text or voice.

[1145] Terminal: Receives the user's emotional data (facial images and voice data), sends the data to the Emotion API for real-time analysis.

[1146] Input: User emotion data (image, voice, text)

[1147] Output: User's emotional state (relaxed, stressed, excited, etc.)

[1148] What it does: The webcam captures the user's face and sends the data in real time to the Emotion API to analyze their emotional state.

[1149] Step 5:

[1150] Server: Based on the analysis results of the emotion engine, the server selects the most suitable interior coordination plan. For example, if the user is looking for relaxation, the server will suggest a calm color scheme and soft furniture.

[1151] Input: Emotional state data and initial analysis data

[1152] Output: Optimal interior coordination plan based on emotional state

[1153] Specific operation: The system automatically adjusts the priority of interior plans based on emotional data, and selects and displays the top plans.

[1154] Step 6:

[1155] Server: Generates a list of furniture items based on the selected interior coordination plan. The list includes detailed information about the items (size, design, material, etc.).

[1156] Input: Interior coordination plan

[1157] Output: A list of furniture items

[1158] Specific operation: Extract relevant furniture information from the database and compile it in list form.

[1159] Step 7:

[1160] Server: For the listed furniture items, search for the best price from multiple shopping sites using APIs and scraping technology. Collect price information and ratings.

[1161] Input: List of furniture items

[1162] Output: Optimal price and rating information for each furniture item

[1163] Specific operation: Use a web scraping tool or API to search for and collect information on relevant items on each shopping site.

[1164] Step 8:

[1165] Server: Calculates the points rewards for each product. Calculates the actual price based on the point reward information from the shopping site or payment platform.

[1166] Input: Furniture item price and special offer information

[1167] Output: Details of point redemption as calculation result

[1168] Specific operation: Calculates the point redemption rate and displays the actual payment amount and redemption points.

[1169] Step 9:

[1170] Server: Sends calculation results and analysis data to the terminal, allowing the user to check the presented information.

[1171] Terminal: Displays interior coordination plans, furniture lists, and point benefit information to the user through an interface.

[1172] Input: Calculation results and analysis data

[1173] Output: Information displayed in the user interface

[1174] Specific operation: The GUI on the terminal displays information in an easy-to-understand format and presents options.

[1175] Step 10:

[1176] User: Check the proposed interior coordination plan and furniture list and select the furniture you want.

[1177] Terminal: Accepts the user's selection and retransmits it to the server.

[1178] Input: User selection information

[1179] Output: Data for selected furniture items

[1180] Specific operation: When the user presses the selection button, the selection information is sent to the server.

[1181] Step 11:

[1182] Server: Generates a purchase link for the furniture item selected by the user and sends it back to the terminal.

[1183] Device: Displays the purchase link to the user and assists them in the purchase process.

[1184] Input: Selection information

[1185] Output: Purchase link for furniture item

[1186] Specific operation: The server generates a purchase link and sends it to the device, where it is displayed to the user and the purchase process proceeds.

[1187] Step 12:

[1188] User: Uses the provided purchase link to purchase furniture through a shopping site or payment platform.

[1189] Server: Follow up on the purchase by sending the user a confirmation of purchase and tracking information.

[1190] Input: Purchase completion information

[1191] Output: Purchase confirmation notice, tracking information

[1192] Specific Operation: After the purchase is completed, the server generates a purchase confirmation notice and sends it to the user's mailbox.

[1193] (Application example 2)

[1194] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1195] Conventional interior coordination systems can suggest furniture based on room layout data and assist in purchasing at the optimal price, but they lack personalized suggestions that take into account the user's emotional state. Furthermore, users cannot actually try out the furniture they selected in the store in real time, making it difficult to visualize the furniture before purchasing. To solve these issues, there is a need for a system that can suggest interior coordination that takes into account the user's emotional state and that uses AR technology to simulate furniture in the real world.

[1196] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1197] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching for the listed furniture at the best price from multiple online shopping sites, means for calculating point benefits associated with purchases from the search results, means for providing information including the calculation results to the user, means for generating a purchase link to the product selected by the user, means for recognizing the user's emotional state, means for selecting an interior coordination plan based on the user's emotional state, and means for displaying a simulation of the real-world arrangement of furniture and interior items using AR technology. This enables personalized suggestions that take the user's emotional state into consideration, and allows for a real-time simulation of the arrangement of selected furniture in a store, making it easier to get an idea of ​​the appearance before purchasing.

[1198] "Room layout data" is spatial information including the structure, size, shape, furniture placement area, window and door positions, and the like of the room.

[1199] "Analysis" refers to the process of analyzing the received room layout data to identify the characteristics of the room.

[1200] An "interior coordination plan" is a plan for furniture placement and color schemes that is suggested based on the room layout data and emotional state.

[1201] "Emotion recognition means" is a function that analyzes the user's emotional state using facial recognition data, etc.

[1202] The "means for listing furniture" is a function for listing necessary furniture items based on the generated interior coordination plan.

[1203] The "means for searching for products at the best price" is a function for searching for the best price for listed furniture items from multiple shopping sites on the Internet.

[1204] The "means for calculating point benefits" is a function for calculating point benefits associated with purchases on a shopping site or payment platform.

[1205] The "means for generating a purchase link" is a function that generates a purchase link to a product selected by the user.

[1206] The "means for selecting an interior coordination plan based on an emotional state" is a function for selecting an optimal interior coordination plan based on emotional data acquired by the emotion recognition means.

[1207] "Means for displaying a simulation of furniture placement using AR technology" refers to a function that uses augmented reality technology to display a simulation of the listed furniture items being placed in the user's room.

[1208] This invention is a system that proposes optimal interior coordination to users based on room layout data and allows them to purchase that furniture at the optimal price. Furthermore, by combining it with an emotion engine, it is possible to recognize the user's emotional state and make suggestions based on that. To implement this invention, the following system is required.

[1209] System Configuration

[1210] The system mainly functions between a server, a terminal, and a user. The user inputs the room layout data using the terminal, and the system proposes an interior coordination plan based on the data and emotional data, and provides optimal purchasing options.

[1211] Hardware and Software Use

[1212] Hardware:

[1213] Facial Recognition Camera

[1214] Smartphone

[1215] software:

[1216] OpenAI API: Used to generate emotion-based furniture recommendations.

[1217] EmotionRecognizer: A library for emotion analysis based on facial recognition data.

[1218] ARRenderer: A renderer that displays furniture simulations using AR technology.

[1219] OpenCV: A library used to capture and display camera footage.

[1220] Furniture API: An API for retrieving furniture data and comparing prices.

[1221] Data processing and data calculation

[1222] 1. The user takes a floor plan and photos of the room and uploads them to the device.

[1223] The device receives this floor plan data and sends it to a server, which temporarily stores it and analyzes it using AI algorithms. The analysis identifies spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[1224] 2. Based on the analysis results, the server generates multiple interior coordination plans, taking into consideration the user's style, color scheme, furniture placement, and other factors.

[1225] 3. The user inputs emotional data into the device. This emotional data can be obtained by facial recognition using a webcam, text input, or voice input. Once the device receives the emotional data, it is sent to the server. The server uses an emotion engine to analyze the emotional data and identify the user's emotional state.

[1226] 4. The server selects the most appropriate interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, it will suggest furniture with calming colors and soft textures.

[1227] 5. The server generates a list of necessary furniture items based on each interior coordination plan. It then uses APIs and scraping technology to search for products at the best prices across multiple shopping sites. It retrieves and lists the lowest price information (price, store, rating, availability, etc.) found for each furniture item. It also calculates the points rewards for each item.

[1228] 6. The server sends information including the calculation results to the terminal, which displays it to the user through an interface. The user reviews the proposed interior coordination plan and furniture list and selects the one they like. The terminal resends the user's selection to the server. The server generates a purchase link for the selected furniture item and sends it to the terminal. The terminal displays the purchase link to the user and supports the purchase process.

[1229] Usage example

[1230] Users use their smartphones in physical stores to upload their room layout data. The server analyzes this data and makes suggestions. The user then uses a camera in the store to provide emotional data, allowing the server to understand the user's desire for relaxation and suggest relaxing interior designs. These interior design suggestions are simulated in real time on the smartphone using AR technology, allowing the user to view them on the spot. Next, furniture with the best price and point benefits is listed, and a purchase link is generated for the furniture selected by the user, allowing for easy purchase procedures.

[1231] Example prompts for generative AI models

[1232] Room data: {"width": 5.0, "length": 6.0, "height": 2.5, "objects": []}

[1233] Emotion: Relaxed

[1234] Suggest furniture and layout:

[1235] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1236] Step 1:

[1237] Users take floor plans and photos of their rooms and upload them to their devices.

[1238] Input: Room floor plan and photos

[1239] Output: Floor plan data saved on the device

[1240] Specific operation: Launch the camera app on your smartphone, take a photo of the floor plan or room, and upload it to the app. The uploaded data is temporarily stored on the device.

[1241] Step 2:

[1242] The terminal receives the room layout data uploaded by the user and transmits it to the server via the interface.

[1243] Input: Floor plan data saved on the device

[1244] Output: Floor plan data sent to the server

[1245] Specific operation: The application on the device sends an HTTP request to the server for the saved floor plan data. The floor plan data is attached to the request.

[1246] Step 3:

[1247] The server temporarily stores the received floor plan data and prepares it for analysis.

[1248] Input: Submitted floor plan data

[1249] Output: Floor plan data saved on the server

[1250] Specific operation: The server receives an HTTP request and temporarily stores the floor plan data in a database. After storing it, it prepares the data so that the analysis program can call it.

[1251] Step 4:

[1252] The server analyzes the stored floor plan data using AI algorithms to identify spatial characteristics such as room size, shape, available furniture placement areas, and the location of windows and doors.

[1253] Input: Floor plan data

[1254] Output: Analysis results identifying room characteristics

[1255] How it works: An AI program on the server analyzes the floor plan data and extracts various room characteristics, including the room's dimensions, wall position, and window and door placement.

[1256] Step 5:

[1257] The server generates a plurality of interior coordination plans based on the analysis results.

[1258] Input: Analysis results

[1259] Output: Interior coordination plan

[1260] How it works: The server's coordinate generator generates different designs and furniture layout plans based on the analysis results, including suggestions based on different styles and themes.

[1261] Step 6:

[1262] The user can input emotion data into the terminal by using face recognition via a webcam, text input, or voice input.

[1263] Input: Emotion data (facial recognition data, text, voice)

[1264] Output: Emotion data stored on the device

[1265] What it does: Users can use their smartphone camera to take a picture of their face so the application can analyze emotional data, or they can use text or voice input methods.

[1266] Step 7:

[1267] The terminal receives the user's emotion data and transmits it to the server.

[1268] Input: Emotion data stored on the device

[1269] Output: Emotion data sent to the server

[1270] Specific operation: The device application sends an HTTP request to the server with emotion data.

[1271] Step 8:

[1272] The server uses an emotion engine to analyze the received emotion data and identify the user's emotional state.

[1273] Input: Emotion data

[1274] Output: Emotional state analysis result

[1275] Specific operation: The emotion engine on the server analyzes the emotion data and identifies the user's emotional state (e.g., relaxed, excited, stressed, etc.).

[1276] Step 9:

[1277] The server selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine.

[1278] Input: Emotional state analysis results, interior coordination plan

[1279] Output: Optimal interior coordination plan based on emotional state

[1280] Specific operation: The server's selection program selects the interior coordination plan that best suits the user's emotional state.

[1281] Step 10:

[1282] The server generates a list of necessary furniture items based on each interior coordination plan.

[1283] Input: Interior coordination plan

[1284] Output: A list of furniture items

[1285] How it works: The server's database and algorithms create a list of the furniture items required based on the selected plan.

[1286] Step 11:

[1287] The server then begins searching for the listed furniture items using APIs and scraping technology to find the best prices on multiple online shopping sites.

[1288] Input: A list of furniture items

[1289] Output: Product information at the best price

[1290] How it works: The server's search algorithm uses each shopping site's API and scraping technology to obtain price information for the listed furniture items.

[1291] Step 12:

[1292] The server retrieves and lists the cheapest product information (price, store, rating, availability, etc.) found for each furniture item.

[1293] Input: Search results

[1294] Output: Information about the cheapest product

[1295] Specific operation: Based on the obtained price information, the server lists the cheapest products and saves them in the database.

[1296] Step 13:

[1297] Based on the search results, the server calculates the point rewards (such as points redemption on shopping sites or payment platforms) for each product.

[1298] Input: Information about the cheapest product

[1299] Output: Point reward information

[1300] Specific operation: The server calculates the point reward information for each product based on the point redemption rules of the shopping site or payment platform.

[1301] Step 14:

[1302] The server transmits the calculated reward information and the search results to the terminal.

[1303] Input: Point reward information, lowest price product information

[1304] Output: Interior coordination plan, furniture list, and point reward information sent to the terminal

[1305] Specific operation: The server sends the calculation results and point reward information together to the terminal so that the user can check them.

[1306] Step 15:

[1307] The terminal displays the interior coordination plan, furniture list, and point benefit information received from the server to the user through an interface.

[1308] Input: Interior coordination plan, furniture list, point reward information

[1309] Output: Information displayed in the interface

[1310] Specific operation: The terminal application displays the information received from the server in the user interface and allows the user to make a selection.

[1311] Step 16:

[1312] The user checks the proposed interior coordination plan and furniture list and selects the one they like.

[1313] Input: User's choice

[1314] Output: User selection information

[1315] Specific operation: The user operates the terminal interface to select the desired item from the proposed coordination plan and furniture list.

[1316] Step 17:

[1317] The terminal accepts the user's selection and retransmits the information to the server.

[1318] Input: User selection information

[1319] Output: Selections sent to the server

[1320] Specific operation: The terminal makes an HTTP request again to send the user's selection information to the server.

[1321] Step 18:

[1322] The server generates a purchase link for the furniture item selected by the user and sends it to the terminal.

[1323] Input: User selection information

[1324] Output: Purchase link

[1325] Specific operation: The server generates a purchase link for the specified furniture item and sends it to the device.

[1326] Step 19:

[1327] The terminal displays a purchase link to the user and assists in the purchase procedure.

[1328] Enter: Purchase Link

[1329] Output: Display link to user

[1330] What happens: The device displays a purchase link in the user interface and helps the user click the link to actually complete the purchase.

[1331] Step 20:

[1332] Users use the provided purchase link to purchase the furniture through a shopping site or payment platform.

[1333] Enter: Purchase Link

[1334] Output: Purchase complete

[1335] What happens: The user clicks on the purchase link, is taken to the shopping site, and completes the purchase.

[1336] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1337] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1338] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1339] [Third embodiment]

[1340] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1341] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1342] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1343] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1344] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1345] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1346] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1347] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1348] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1349] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1350] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1351] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1352] This invention relates to a system that proposes optimal interior coordination to users based on room layout data and also allows them to purchase the furniture at the optimal price. The specific operations performed in the processing of each step are described below.

[1353] System Configuration

[1354] The system mainly works between a server, a terminal, and a user. The user inputs the layout data of the room using the terminal, and the server uses that data to suggest interior coordination and provide optimal purchasing options.

[1355] Program processing

[1356] 1. Enter and submit room layout data

[1357] User: Takes a floor plan and photographs of the room, and uploads them to the system via a device.

[1358] Terminal: Sends the uploaded data to the server.

[1359] Server: Temporarily stores received data and prepares it for analysis.

[1360] 2. Analysis of room layout data

[1361] Server: Analyzes the received floor plan data using an AI algorithm to identify spatial characteristics such as room size, shape, available furniture placement areas, and the location of windows and doors.

[1362] Example: If a floor plan is divided into a living room, bedroom, and kitchen, the system automatically recognizes the area and shape of each room, as well as the space available for furniture placement.

[1363] 3. Creating an interior coordination plan

[1364] Server: Based on the analysis results, it generates multiple interior coordination plans according to the purpose of use of the space and the user's preferences.

[1365] Example: For a living room, suggestions include a Scandinavian-style furniture arrangement plan and a modern furniture arrangement plan.

[1366] 4. List your furniture items

[1367] Server: Generates a list of required furniture items based on the proposed interior coordination plan.

[1368] Example: Select key items for each category, such as sofas, dining tables, and beds.

[1369] 5. Find the best priced furniture

[1370] Server: Searches multiple online shopping sites for the best price for the listed furniture items.

[1371] Example: Search for "L-shaped sofa" on multiple shopping sites and determine the best product based on price, reviews, delivery terms, etc.

[1372] 6. Calculation of points rewards

[1373] Server: Calculates the points reward for each product based on the search results.

[1374] Example: Calculate the reward points when purchasing using a combination of Yahoo! Shopping and PayPay.

[1375] 7. Providing Information to the User Interface

[1376] Server: Sends calculated reward information and search results to the device.

[1377] Terminal: Displays proposed interior plans and product lists to the user through a user interface.

[1378] 8. Generate Purchase Link

[1379] User: Select the best from the proposed interior plans and product list.

[1380] Server: Generates a purchase link for the product selected by the user and sends it to the device.

[1381] Device: Show the purchase link to the user and assist them in the purchase process.

[1382] Specific examples

[1383] A user wants to redecorate their living room, so they upload a floor plan. The server analyzes the living room, recognizes its size and shape, and suggests furniture placement. It then searches the internet for the best prices for each furniture item and calculates the combination that maximizes point rewards. Finally, it generates a purchase link and presents it to the user, allowing them to easily purchase the furniture.

[1384] This system allows users to effortlessly coordinate their interiors and purchase furniture economically all in one place.

[1385] The processing flow will be explained below.

[1386] Step 1:

[1387] User: Take a floor plan or photo of the room and upload it to the device.

[1388] Step 2:

[1389] Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[1390] Step 3:

[1391] Server: Temporarily stores the received floor plan data and prepares it for data analysis.

[1392] Step 4:

[1393] Server: Analyzes stored floor plan data using AI algorithms to identify spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[1394] Step 5:

[1395] Server: Generates multiple interior coordination plans based on the analysis results, proposing plans that take into account style, color scheme, furniture placement, etc.

[1396] Step 6:

[1397] Server: Generates a list of required furniture items based on each interior coordination plan.

[1398] Step 7:

[1399] Server: Initiates a search for the listed furniture items using APIs and scraping technology to find the best prices from multiple shopping sites.

[1400] Step 8:

[1401] Server: Retrieves and lists the cheapest price information (price, retailer, ratings, availability, etc.) found for each furniture item.

[1402] Step 9:

[1403] Server: Based on the search results, calculate the points rewards for each product (such as Yahoo! Shopping or PayPay points).

[1404] Step 10:

[1405] Server: Sends the interior coordination plan and furniture list including the calculation results to the terminal.

[1406] Step 11:

[1407] Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through an interface.

[1408] Step 12:

[1409] User: Review the proposed interior coordination plan and furniture list and select the one they like.

[1410] Step 13:

[1411] Terminal: Accepts the user's selection and resends the information to the server.

[1412] Step 14:

[1413] Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[1414] Step 15:

[1415] Device: Show the purchase link to the user and assist them in the purchase process.

[1416] Step 16:

[1417] User: Use the provided purchase link to purchase furniture through a shopping site or PayPay.

[1418] Step 17:

[1419] Server: Follow up with the user after the purchase by sending them a confirmation of the purchase and tracking information.

[1420] Example 1

[1421] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1422] Conventional interior coordination systems lacked the functionality to suggest optimal furniture placement to users. Furthermore, providing information on how to purchase furniture at the best price and calculating point rewards associated with purchases were often done manually, which was time-consuming for users. Furthermore, the systems lacked the means to safely handle users' personal information, leaving them at risk of data theft and unauthorized access.

[1423] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1424] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching multiple online shopping sites for products at the best price for the listed furniture, means for calculating points related to purchases from the search results, means for providing the user with information including the calculation results, means for generating a purchase link to the product selected by the user, and means for encrypting and transmitting data to protect the user's privacy. This allows the user to effortlessly coordinate an optimal interior and purchase furniture economically all in one place.

[1425] "Room layout data" refers to data provided by users that includes information such as the dimensions and shape of the room, the area in which furniture can be placed, and the positions of windows and doors.

[1426] The "receiving means" is a system or device for receiving room layout data from a user.

[1427] "Means for analysis" refers to a system or device that analyzes the received room layout data using an AI algorithm or the like to identify the structure and characteristics of the room.

[1428] A "means for identifying features" is a system or device that has the ability to extract important spatial information from the analysis results, such as the size and shape of the room, the area where furniture can be placed, and the location of windows and doors.

[1429] An "interior coordination plan" is data that shows furniture placement and design plans proposed based on the user's preferences and room characteristics.

[1430] The "generating means" is a system or device for creating a plurality of interior coordination plans based on the analysis results.

[1431] The "listing means" is a system or device having a function for creating a list of necessary furniture and decorative items based on the generated interior coordination plan.

[1432] The "means for searching for products at the best price" is a system or device that has the function of searching for the most favorable priced product for the listed furniture from multiple online shopping sites.

[1433] The "means for calculating point benefits" is a system or device that has the function of calculating the total points redeemed based on the point redemption information provided by multiple shopping sites on the Internet based on search results.

[1434] The "means for providing information including calculation results to users" refers to a system or device for notifying users of search results including point redemption information and visually displaying them.

[1435] The "means for generating a purchase link" refers to a system or device for creating a link that allows a user to directly purchase the product selected by the user on an online shopping site.

[1436] "Means for encryption and transmission" refers to a system or device that encrypts data such as floor plan data and user information and communicates them securely in order to protect user privacy.

[1437] This invention relates to a system that proposes optimal interior coordination to users based on room layout data and allows them to purchase the furniture at the optimal price. This system functions in cooperation with a server, terminals, and users.

[1438] System Configuration

[1439] The user inputs the layout data of the room using a terminal, and the server uses that data to propose an interior coordination plan and also provides optimal purchasing options. In terms of specific hardware, the user's terminal can be a smartphone, tablet, or PC, while the server uses a high-performance computer system.

[1440] Entering and Submitting Data

[1441] User

[1442] Users take photos of the room floor plan and other images using the camera on their smartphone or PC, and upload the data to the system using the following prompts:

[1443] "I uploaded the living room floor plan."

[1444] "I sent you the bedroom floor plan."

[1445] Terminal

[1446] The terminal temporarily stores the data entered by the user and transmits it to the server using a secure communication protocol (HTTPS).

[1447] server

[1448] The server temporarily stores the received data and checks the data for consistency and format. If any errors are found, the user is notified and asked to re-upload.

[1449] Data analysis

[1450] server

[1451] The server uses AI algorithms (e.g., Convolutional Neural Networks) to analyze the data it receives, identifying important spatial features such as the room's size, shape, available furniture area, and the location of windows and doors.

[1452] As a specific example, if a floor plan is divided into a living room, bedroom, and kitchen, the area and shape of each room, as well as the space available for furniture placement, are automatically recognized using libraries such as HTML5 Canvas and OpenCV.

[1453] Coordination plan generation

[1454] server

[1455] The server generates multiple interior coordination plans based on the above analysis results, tailored to the user's preferences and lifestyle. Using a specific generation AI model, the server generates plans using the following prompts:

[1456] "Please suggest a Scandinavian-style interior design for the living room."

[1457] "Generate a modern bedroom furniture arrangement plan."

[1458] The generated plans are offered for selection by the user.

[1459] Listing furniture items

[1460] server

[1461] Based on the plan generated, create a list of the furniture items you'll need, including items in each major category, such as a sofa, dining table, and bed.

[1462] Find the best price

[1463] server

[1464] The server searches multiple online shopping sites in real time for the best price for the listed furniture, taking into account not only price but also ratings, delivery terms, and other factors.

[1465] Points Reward Calculation

[1466] server

[1467] Based on the search results, the points rewards for each product are calculated. For example, the total points are calculated based on the point reward rates on Yahoo! Shopping and other online platforms.

[1468] Providing information

[1469] server

[1470] Information including the calculation results is sent to the terminal, which then visually displays it so that the user can easily understand it.

[1471] Generate purchase links

[1472] User

[1473] The user selects the most suitable item from the proposed interior plans and product list.

[1474] server

[1475] The server generates a purchase link for the product selected by the user and sends it to the terminal, which displays the link to the user and assists in the purchase process.

[1476] Data encryption

[1477] Terminals and Servers

[1478] To protect user privacy, all data transmitted is encrypted using Transport Layer Security (TLS), which protects user information from data theft and unauthorized access.

[1479] As described above, this system minimizes the user's effort and enables optimal interior coordination and economical furniture purchases.

[1480] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1481] Step 1:

[1482] input

[1483] User: Take a floor plan or photo of a room.

[1484] Users: Upload data via a dedicated application or web portal.

[1485] output

[1486] Device: Temporarily stores floor plans and photo data.

[1487] Terminal: Sends data to the server using a secure communication protocol (HTTPS).

[1488] Specific actions

[1489] Users take a picture of the room floor plan using the camera on their smartphone or PC.

[1490] The captured data is saved in the application and sent to the server.

[1491] Step 2:

[1492] input

[1493] Terminal: Receives floor plan data from the user.

[1494] Server: Receives floor plan data sent from the device.

[1495] output

[1496] Server: Temporarily stores the received data and checks its structural integrity.

[1497] Server: If the data is in the correct format, prepare it for analysis. If it is not, send an error notification to the user.

[1498] Specific actions

[1499] The server checks the integrity of the data and notifies the user if there are any discrepancies.

[1500] If the data format is correct, proceed to the next analysis step.

[1501] Step 3:

[1502] input

[1503] Server: Received floor plan data

[1504] output

[1505] Server: Analyzes floor plan data and identifies room size, shape, available furniture placement area, and window and door locations.

[1506] Specific actions

[1507] The server analyzes the floor plan data using an AI algorithm (e.g., Convolutional Neural Network).

[1508] Identify room boundaries, dimensions, and functional areas and store them in a database.

[1509] Step 4:

[1510] input

[1511] Server: Analysis results (room size, shape, furniture placement area, etc.)

[1512] output

[1513] Server: Multiple interior coordination plans based on user preferences

[1514] Specific actions

[1515] The server uses a generative AI model to generate a coordination plan based on the prompt sentence.

[1516] Example: "Generate a Scandinavian furniture arrangement plan for my living room."

[1517] Organize the proposed plans and select the most suitable one.

[1518] Step 5:

[1519] input

[1520] Server: Generated interior coordination plan

[1521] output

[1522] Server: List of required furniture items

[1523] Specific actions

[1524] The server lists the necessary furniture items based on the coordination plan.

[1525] Select key items for each category, such as sofas and dining tables.

[1526] Step 6:

[1527] input

[1528] Server: List of furniture items

[1529] output

[1530] Server: A list of products with the best prices from multiple shopping sites on the Internet

[1531] Specific actions

[1532] The server crawls multiple shopping sites and compares prices.

[1533] Example: Search for "L-shaped sofa" across multiple sites to identify the best products.

[1534] Product ratings and delivery terms are also taken into consideration.

[1535] Step 7:

[1536] input

[1537] Server: Price comparison results

[1538] output

[1539] Server: Calculation result of point benefits for each product

[1540] Specific actions

[1541] The server calculates the point rewards for each product based on the point redemption rate.

[1542] Example: Calculating Yahoo! Shopping reward points.

[1543] Step 8:

[1544] input

[1545] Server: Calculated point reward information

[1546] Terminal: Information received from the server

[1547] output

[1548] Terminal: Displays interior plans and product lists in a user-friendly format

[1549] Specific actions

[1550] The terminal retrieves the calculated reward information and displays it in a graphical interface.

[1551] Product images, price information, point redemption information, and more can be viewed at a glance.

[1552] Step 9:

[1553] input

[1554] User: Select the furniture item they want from the list of suggested items

[1555] output

[1556] Server: Purchase link for selected product

[1557] Specific actions

[1558] The server generates a purchase link for the furniture item selected by the user.

[1559] Send a purchase link to the device to allow users to easily make a purchase.

[1560] By following the above steps, users can efficiently and easily coordinate their interiors optimally and purchase furniture economically.

[1561] (Application example 1)

[1562] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1563] Currently, when coordinating interiors, it is difficult to select and arrange the optimal furniture to fit the floor plan. Furthermore, there is a lack of ways to purchase furniture at the best price or to virtually check what the interior will look like after purchase. This requires users to spend a lot of time and effort, making it difficult to achieve effective interior coordination.

[1564] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1565] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching multiple online shopping sites for products at the best prices for the listed furniture, means for calculating point benefits associated with purchases from the search results, means for providing the user with information including the calculation results, means for generating purchase links to products selected by the user, means for the user to 3D scan the floor plan data, and means for confirming the interior coordination plan in a virtual space using the 3D scanned data. This allows the user to efficiently and economically realize optimal interior coordination and virtually confirm it.

[1566] "Room layout data" is information that indicates the layout, size, and shape of a physical space such as a user's residence or office.

[1567] "Means for receiving" refers to devices or software that have the functionality to allow the system to acquire data sent from the user.

[1568] "Means for analysis" refers to algorithms or software that analyze the received data and identify the structure and characteristics of the space.

[1569] An "interior coordination plan" is a specific proposal for the layout and design of furniture and decorations to be placed in a room.

[1570] The "listing means" refers to a device or software that has the function of selecting the necessary furniture and products based on the interior coordination plan.

[1571] "Internet shopping site" refers to a website where users can search for and purchase products online.

[1572] A "means for searching for products at the best price" is an algorithm or software that searches multiple shopping sites for products that meet your criteria.

[1573] The "means for calculating point benefits" refers to a device or software that has the function of calculating points or discounts that can be obtained at the time of purchase.

[1574] "Means for providing information to users" refers to the interface or software for displaying calculation results and proposals to users.

[1575] The "means for generating a purchase link" refers to a device or software that has the function of creating a link for purchasing the product selected by the user.

[1576] "3D scanning means" refers to devices and software for digitizing physical spaces as three-dimensional data.

[1577] "Means for checking in a virtual space" refers to devices or software that have the function of allowing users to visually check interior coordination plans using three-dimensional models.

[1578] The present invention relates to a system that allows a user to propose optimal interior coordination based on room layout data, and detailed embodiments thereof will be described.

[1579] System Configuration

[1580] The system mainly functions between the server, the terminal, and the user. The system receives room layout data and 3D scan data from the user and proposes interior coordination in the virtual space. Specifically, it works as follows:

[1581] Hardware and Software

[1582] Hardware:

[1583] A smartphone or head-mounted display (e.g., Oculus Quest 2)

[1584] Server (Cloud server or on-premise server)

[1585] software:

[1586] Unity, Vuforia (AR technology), OpenCV

[1587] Python and Flask (server-side processing)

[1588] MongoDB (data storage)

[1589] Processing flow

[1590] Data reception and analysis

[1591] 1. User:

[1592] Use a smartphone or head-mounted display to 3D scan the layout of a room.

[1593] The acquired data is sent from the terminal to the server.

[1594] 2. Server:

[1595] The server analyzes the received 3D scan data using OpenCV to determine the size, shape, and area where furniture can be placed in the room.

[1596] The analysis results are saved in MongoDB.

[1597] Generate interior coordination plans

[1598] 3. Server:

[1599] Based on the analysis results, a Python script is used to generate multiple interior coordination plans, such as Scandinavian and modern styles.

[1600] Check in virtual space

[1601] 4. User:

[1602] Using Unity and Vuforia, participants can visually check 3D models of proposed interior coordination plans in a virtual space using a head-mounted display or smartphone.

[1603] Product listing and price search

[1604] 5. Server:

[1605] The system creates a list of furniture items based on the generated interior coordination plan and uses a Python script to search multiple online shopping sites for the best prices.

[1606] Calculate points rewards for each product based on search results.

[1607] Purchase links and information

[1608] 6. Server:

[1609] The search results and calculated point reward information are sent to the terminal and displayed to the user.

[1610] A purchase link is generated for the product selected by the user and sent to the terminal.

[1611] 7. Users:

[1612] Check the proposed interior plans and product list and select the most suitable products.

[1613] Easily purchase products through our purchase links.

[1614] Prompt Sentence Examples

[1615] The invention provides a virtual interior coordination assistant application that allows users to send scanned data of a room to a server and easily coordinate the optimal interior and compare prices.

[1616] As a concrete example, a user considers redecorating their bedroom and scans the room using a head-mounted display. The system analyzes the size and shape of the room and proposes interior plans in both Scandinavian and modern styles. The user can then view the virtual 3D models and purchase furniture that suits their tastes at the best price.

[1617] This system allows users to efficiently coordinate their interiors optimally and purchase furniture at economical prices.

[1618] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1619] Step 1:

[1620] Users use a smartphone or head-mounted display to 3D scan the layout of a room, and the scanned data is then sent from the device to a server.

[1621] Input: 3D scan of a room.

[1622] Processing: The device receives the 3D scan data taken by the user, converts it into the appropriate format, and sends it to the server.

[1623] Output: 3D scan data sent to the server.

[1624] Step 2:

[1625] The server analyzes the received 3D scan data to determine the size, shape, and available furniture area of ​​the room using OpenCV.

[1626] Input: 3D scan data.

[1627] Processing: Using OpenCV, the server analyzes the 3D scan data and extracts spatial features (size, shape, furniture placement area), applying image processing techniques such as edge detection and shape analysis.

[1628] Output: Data about the room size, shape, and available furniture area.

[1629] Step 3:

[1630] The server generates multiple interior coordination plans based on the analysis results, using a Python script.

[1631] Input: Data about the room size, shape, and available furniture area.

[1632] Processing: Using a Python script, multiple interior coordination plans (e.g., Scandinavian style, modern style) are generated based on the analysis results. Various plans are created by referencing past design data and user preferences.

[1633] Output: Interior coordination plan data.

[1634] Step 4:

[1635] Using Unity and Vuforia, users can visually view 3D models of proposed interior coordination plans in a virtual space using a head-mounted display or smartphone.

[1636] Input: Interior coordination plan data.

[1637] Processing: Using Unity and Vuforia, the interior coordination plan is created as a 3D model and displayed in a user interface, allowing users to visually inspect the model and perform operations such as rotating and scaling.

[1638] Output: A virtual interior coordination that can be visually confirmed by the user.

[1639] Step 5:

[1640] The server lists furniture items based on the interior coordination plan and uses a Python script to search multiple shopping sites on the Internet for the best priced items.

[1641] Input: Interior coordination plan data.

[1642] Processing: The server crawls the web for the listed furniture items, collects information from multiple shopping sites, and compares and selects the best options based on price, reviews, delivery terms, etc.

[1643] Output: A list of the best purchasing options for each furniture item.

[1644] Step 6:

[1645] The server calculates the points reward for each product based on the search results.

[1646] Enter: the best buying options list.

[1647] Processing: The server obtains the point reward information of each shopping site and calculates the number of points that can be earned at the time of purchase. This is done to calculate the economic efficiency of the purchase.

[1648] Output: A list of optimal purchase options with point rewards.

[1649] Step 7:

[1650] The server generates the calculation results and a link to purchase the furniture, and sends them to the device. The user can then use their smartphone or head-mounted display to check the interior plan and product list and select the most suitable product. The product can then be easily purchased using the purchase link.

[1651] Enter: a list of the best purchasing options, including points rewards.

[1652] Processing: Organize the calculated reward information and product list, generate data to display in the user interface, generate a purchase link for the product selected by the user, and send it to the device.

[1653] Output: Interior plans and product list with purchase links.

[1654] This series of processes allows users to efficiently and economically create optimal interior coordination and easily purchase furniture while checking it virtually.

[1655] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1656] This invention combines an emotion engine with a system that proposes optimal interior coordination to users based on room layout data and then allows them to purchase that furniture at the optimal price. The emotion engine recognizes the user's emotional state and makes suggestions based on that.

[1657] System Configuration

[1658] The system mainly works between the server, the terminal, and the user. The user inputs the room layout data using the terminal, and the server proposes interior coordination based on that data and the user's emotional data, and provides optimal purchasing options.

[1659] Program processing

[1660] Enter and submit room layout data

[1661] 1. User: Take a floor plan or photo of the room and upload it to the device.

[1662] 2. Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[1663] 3. Server: Temporarily stores the received data and prepares it for analysis.

[1664] Analysis of room layout data

[1665] 4. Server: The stored floor plan data is analyzed using AI algorithms to identify spatial characteristics such as room size, shape, furniture placement area, and window and door locations.

[1666] Generate interior coordination plans

[1667] 5. Server: Based on the analysis results, it generates multiple interior coordination plans, taking into account style, color scheme, furniture placement, etc.

[1668] Emotion data input and analysis

[1669] 6. User: Emotional data can be input to the device using facial recognition via a webcam, text input, or voice input.

[1670] 7. Terminal: Receives the user's emotion data and sends it to the server.

[1671] 8. Server: Analyzes the received emotion data using the emotion engine and identifies the user's emotional state.

[1672] Generate personalized proposals

[1673] 9. Server: Selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, it will suggest furniture with calming colors and soft textures.

[1674] Listing furniture items

[1675] 10. Server: Generates a list of required furniture items based on each interior coordination plan.

[1676] Find the best priced furniture

[1677] 11. Server: Initiates a search for the listed furniture items using APIs and scraping techniques to find the best prices from multiple shopping sites.

[1678] 12. Server: Retrieves and lists the cheapest price information (price, retailer, rating, availability, etc.) found for each furniture item.

[1679] Points Reward Calculation

[1680] 13. Server: Based on the search results, calculates the points rewards for each product (such as points redemption on shopping sites or payment platforms).

[1681] Providing information to the user interface

[1682] 14. Server: Sends the calculated reward information and search results to the device.

[1683] 15. Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through the interface.

[1684] Generate purchase links

[1685] 16. User: Review the proposed interior coordination plan and furniture list and select the one they like best.

[1686] 17. Terminal: Accepts the user's selection and resends the information to the server.

[1687] 18. Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[1688] 19. Device: Display the purchase link to the user and assist with the purchase process.

[1689] Purchasing and Follow-up

[1690] 20. User: Uses the provided purchase link to purchase furniture through a shopping site or payment platform.

[1691] 21. Server: Follow up on the purchase by sending the user a confirmation of the purchase and tracking information.

[1692] Specific examples

[1693] A user wants to redecorate their living room and uploads a floor plan. The server analyzes the living room, recognizes its size and shape, and suggests furniture placement. The user then provides emotional data via webcam, revealing that they are looking for a relaxing space. Based on this information, the server suggests furniture with calming color schemes and interior design that creates a relaxing effect. The server searches multiple shopping sites for the best prices and point rewards for the furniture and provides them to the user. Once the user selects the furniture, a purchase link is generated to help them easily complete the purchase process.

[1694] This system allows users to coordinate their interiors to best suit their emotional state and makes hassle-free and economical furniture purchases all in one go.

[1695] The processing flow will be explained below.

[1696] Step 1:

[1697] User: Take a floor plan or photo of the room and upload it to the device.

[1698] Step 2:

[1699] Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[1700] Step 3:

[1701] Server: Temporarily stores the received floor plan data and prepares it for data analysis.

[1702] Step 4:

[1703] Server: Analyzes stored floor plan data using AI algorithms to identify spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[1704] Step 5:

[1705] Server: Generates multiple interior coordination plans based on the analysis results, proposing plans that take into account style, color scheme, furniture placement, etc.

[1706] Step 6:

[1707] User: Enters emotion data into the device using face recognition via a webcam, text input, or voice input.

[1708] Step 7:

[1709] Terminal: Receives the user's emotion data and sends the emotion data to the server.

[1710] Step 8:

[1711] Server: Analyzes the received emotional data using the emotion engine and identifies the user's emotional state. If the user wants to relax, it will be labeled as "Relaxed," and if the user wants to be energized, it will be labeled as "Energy."

[1712] Step 9:

[1713] Server: Selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, the server will suggest furniture with calming colors and soft textures.

[1714] Step 10:

[1715] Server: Generates a list of required furniture items based on each interior coordination plan.

[1716] Step 11:

[1717] Server: Initiates a search for the listed furniture items using APIs and scraping technology to find the best prices from multiple shopping sites.

[1718] Step 12:

[1719] Server: Retrieves and lists the cheapest price information (price, retailer, ratings, availability, etc.) found for each furniture item.

[1720] Step 13:

[1721] Server: Based on the search results, calculates the reward points for each product (such as points redemption from shopping sites or payment platforms).

[1722] Step 14:

[1723] Server: Sends the interior coordination plan and furniture list including the calculation results to the terminal.

[1724] Step 15:

[1725] Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through an interface.

[1726] Step 16:

[1727] User: Review the proposed interior coordination plan and furniture list and select the one they like.

[1728] Step 17:

[1729] Terminal: Accepts the user's selection and resends the information to the server.

[1730] Step 18:

[1731] Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[1732] Step 19:

[1733] Device: Show the purchase link to the user and assist them in the purchase process.

[1734] Step 20:

[1735] User: Purchases furniture through a shopping site or payment platform using the provided purchase link.

[1736] Step 21:

[1737] Server: Follow up with the user after the purchase by sending them a confirmation of the purchase and tracking information.

[1738] Example 2

[1739] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1740] Conventional interior coordination suggestion systems have difficulty making personalized suggestions that take into account the user's emotional state. Furthermore, searching for furniture prices and presenting point benefits is time-consuming, and the system provides insufficient support for users to make optimal choices. This makes it difficult to achieve interior coordination that satisfies users and to efficiently purchase furniture.

[1741] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for acquiring and analyzing the user's emotional state, means for selecting an appropriate interior coordination plan based on the acquired emotional state, means for listing furniture based on the generated interior coordination plan, means for searching multiple shopping sites on the Internet for products at the best prices for the listed furniture, means for calculating point benefits related to purchases from the search results, means for providing the user with information including the calculation results, and means for generating purchase links to products selected by the user. This allows the server to suggest optimal interior coordination based on the user's emotional state, enabling efficient and economical furniture purchases.

[1742] "Room layout data" is numerical and graphical information that indicates the shape and dimensions of a room, the area in which furniture can be placed, the positions of windows and doors, and so on.

[1743] An "interior coordination plan" is a proposal for the interior design of a room, including the style, color scheme, furniture placement, and decorative items.

[1744] "Emotional state" is information that indicates the user's mood and state of mind, and is primarily obtained through facial recognition, voice analysis, text input, etc.

[1745] A "furniture list" is a list of necessary furniture items based on a specific interior coordination plan, including detailed information about each item.

[1746] The "best price" is the lowest price obtained from multiple shopping sites on the Internet for the furniture item specified by the user.

[1747] "Point benefits" are points or discount information provided by shopping sites or payment platforms that are given back to users when they make a purchase.

[1748] A "purchase link" is a direct URL or hyperlink to an online shopping site that allows a user to purchase a specified furniture item.

[1749] This invention is a system that proposes optimal interior coordination to users based on room layout data. This system makes personalized suggestions based on the user's emotional state, and also allows the user to purchase furniture at the optimal price based on those suggestions.

[1750] System Configuration

[1751] This system mainly functions between a server, a terminal, and a user. The user inputs room layout data using the terminal, and the server proposes interior coordination based on that data and the user's emotional data, and provides optimal purchasing options.

[1752] Hardware and software used

[1753] Devices: Smartphone, tablet, PC (with webcam)

[1754] Server: Cloud server (e.g. Amazon Web Services)

[1755] AI algorithm: TensorFlow (for image analysis)

[1756] Emotion analysis engine: Emotion API (face recognition, voice analysis)

[1757] Database: SQL database (for managing customer information and furniture lists)

[1758] API: Web scraping tool (for product price search)

[1759] Program processing

[1760] Enter and submit room layout data

[1761] The user uses the device to take floor plans and photos of the room and uploads them to the device through a dedicated application. The device receives the data uploaded by the user and sends it to the server.

[1762] Analysis of room layout data

[1763] The server analyzes the received floor plan data using an AI algorithm (e.g., TensorFlow) to identify spatial characteristics such as room size, shape, and available furniture placement areas, as well as the location of windows and doors.

[1764] Generate interior coordination plans

[1765] The server generates multiple interior coordination plans based on the analysis results, including different styles, color schemes, and furniture placements.

[1766] Emotion data input and analysis

[1767] Users can input emotion data into their devices using facial recognition via a webcam, text input, or voice input. The device receives the user's emotion data and sends it to a server. The server then uses an emotion analysis engine (e.g., Emotion API) to identify the user's emotional state.

[1768] Generate personalized proposals

[1769] The server selects the most appropriate interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking to relax, it will suggest furniture with a calm color scheme and soft textures.

[1770] Listing furniture items

[1771] The server generates a list of necessary furniture items based on the selected interior plan, including detailed information about each piece (size, design, material, etc.).

[1772] Find the best priced furniture

[1773] The server searches for the best priced products for the listed furniture items from multiple shopping sites using APIs and scraping technology, and obtains the best prices and ratings for the furniture based on the search results.

[1774] Points Reward Calculation

[1775] The server calculates the points rewards (points redeemed by shopping sites and payment platforms) for each product, allowing users to make the most economical overall choice.

[1776] Providing information to the user interface

[1777] The server transmits the calculated reward information and search results to the terminal, which displays the information to the user through an interface.

[1778] Generate purchase links

[1779] The user checks the presented interior coordination plan and furniture list and selects the furniture they want. The terminal retransmits the user's selection to the server. The server generates a purchase link for the furniture item selected by the user and transmits it again to the terminal. The terminal displays the purchase link to the user and supports the purchase procedure.

[1780] Purchasing and Follow-up

[1781] The user uses the provided purchase link to purchase the furniture through a shopping site or payment platform, and the server then sends the user a confirmation of the purchase and tracking information as follow-up after the purchase.

[1782] Specific operation example

[1783] For example, if a user is thinking about redecorating their living room, they first upload a floor plan. The server analyzes the floor plan and recognizes the size and shape of the living room. Next, the user provides emotional data via webcam, and it is determined that they are looking for a relaxing space. Based on this information, the server suggests calming color schemes and interior design that has a relaxing effect. It also searches multiple shopping sites for the best prices and point rewards for furniture and provides them to the user. The server then generates a purchase link for the furniture selected by the user, facilitating a simple purchasing process.

[1784] Prompt Sentence Examples

[1785] I'm thinking of redecorating my living room. Please suggest the best interior design based on this floor plan. I want to create a relaxing space.

[1786] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1787] Step 1:

[1788] User: Takes a photo or a floor plan of the room and uploads it to the device via a dedicated application. The request includes the floor plan and photo data. For example, a user takes a photo of the living room floor plan with their smartphone and presses the "upload" button.

[1789] Terminal: Receives floor plan data uploaded by the user, checks the data format (JPEG or PNG), and sends it to the server. Once the data format check is complete, it is sent to the server.

[1790] Specific operation: When the user presses the "Upload floor plan" button, the image data of the floor plan is saved on the device and transferred to the server.

[1791] Step 2:

[1792] Server: The received floor plan data is temporarily stored in cloud storage (e.g., Amazon S3), and then an AI algorithm (TensorFlow) is invoked to analyze it.

[1793] Input: Received image data of floor plans and photos

[1794] Output: Analytical data including room size, shape, available furniture area, and spatial characteristics such as window and door locations

[1795] Specific operation: The server passes the stored floor plan to an AI algorithm, which performs edge detection and object recognition. The analysis results are then stored in a database.

[1796] Step 3:

[1797] Server: Based on the analysis results, the server executes the interior coordination plan generation program. Here, it references several interior style templates from the database and selects a plan that suits the user's room.

[1798] Input: Room analysis data

[1799] Output: Multiple interior coordination plans (style, color scheme, furniture placement)

[1800] Specific operation: Based on the analysis data, multiple optimal templates are extracted and images and text information for presentations are generated.

[1801] Step 4:

[1802] User: Activate the device's webcam and use the facial recognition system to input emotion data, or choose to input text or voice.

[1803] Terminal: Receives the user's emotional data (facial images and voice data), sends the data to the Emotion API for real-time analysis.

[1804] Input: User emotion data (image, voice, text)

[1805] Output: User's emotional state (relaxed, stressed, excited, etc.)

[1806] What it does: The webcam captures the user's face and sends the data in real time to the Emotion API to analyze their emotional state.

[1807] Step 5:

[1808] Server: Based on the analysis results of the emotion engine, the server selects the most suitable interior coordination plan. For example, if the user is looking for relaxation, the server will suggest a calm color scheme and soft furniture.

[1809] Input: Emotional state data and initial analysis data

[1810] Output: Optimal interior coordination plan based on emotional state

[1811] Specific operation: The system automatically adjusts the priority of interior plans based on emotional data, and selects and displays the top plans.

[1812] Step 6:

[1813] Server: Generates a list of furniture items based on the selected interior coordination plan. The list includes detailed information about the items (size, design, material, etc.).

[1814] Input: Interior coordination plan

[1815] Output: A list of furniture items

[1816] Specific operation: Extract relevant furniture information from the database and compile it in list form.

[1817] Step 7:

[1818] Server: For the listed furniture items, search for the best price from multiple shopping sites using APIs and scraping technology. Collect price information and ratings.

[1819] Input: List of furniture items

[1820] Output: Optimal price and rating information for each furniture item

[1821] Specific operation: Use a web scraping tool or API to search for and collect information on relevant items on each shopping site.

[1822] Step 8:

[1823] Server: Calculates the points rewards for each product. Calculates the actual price based on the point reward information from the shopping site or payment platform.

[1824] Input: Furniture item price and special offer information

[1825] Output: Details of point redemption as calculation result

[1826] Specific operation: Calculates the point redemption rate and displays the actual payment amount and redemption points.

[1827] Step 9:

[1828] Server: Sends calculation results and analysis data to the terminal, allowing the user to check the presented information.

[1829] Terminal: Displays interior coordination plans, furniture lists, and point benefit information to the user through an interface.

[1830] Input: Calculation results and analysis data

[1831] Output: Information displayed in the user interface

[1832] Specific operation: The GUI on the terminal displays information in an easy-to-understand format and presents options.

[1833] Step 10:

[1834] User: Check the proposed interior coordination plan and furniture list and select the furniture you want.

[1835] Terminal: Accepts the user's selection and retransmits it to the server.

[1836] Input: User selection information

[1837] Output: Data for selected furniture items

[1838] Specific operation: When the user presses the selection button, the selection information is sent to the server.

[1839] Step 11:

[1840] Server: Generates a purchase link for the furniture item selected by the user and sends it back to the terminal.

[1841] Device: Displays the purchase link to the user and assists them in the purchase process.

[1842] Input: Selection information

[1843] Output: Purchase link for furniture item

[1844] Specific operation: The server generates a purchase link and sends it to the device, where it is displayed to the user and the purchase process proceeds.

[1845] Step 12:

[1846] User: Uses the provided purchase link to purchase furniture through a shopping site or payment platform.

[1847] Server: Follow up on the purchase by sending the user a confirmation of purchase and tracking information.

[1848] Input: Purchase completion information

[1849] Output: Purchase confirmation notice, tracking information

[1850] Specific Operation: After the purchase is completed, the server generates a purchase confirmation notice and sends it to the user's mailbox.

[1851] (Application example 2)

[1852] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1853] Conventional interior coordination systems can suggest furniture based on room layout data and assist in purchasing at the optimal price, but they lack personalized suggestions that take into account the user's emotional state. Furthermore, users cannot actually try out the furniture they selected in the store in real time, making it difficult to visualize the furniture before purchasing. To solve these issues, there is a need for a system that can suggest interior coordination that takes into account the user's emotional state and that uses AR technology to simulate furniture in the real world.

[1854] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1855] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching for the listed furniture at the best price from multiple online shopping sites, means for calculating point benefits associated with purchases from the search results, means for providing information including the calculation results to the user, means for generating a purchase link to the product selected by the user, means for recognizing the user's emotional state, means for selecting an interior coordination plan based on the user's emotional state, and means for displaying a simulation of the real-world arrangement of furniture and interior items using AR technology. This enables personalized suggestions that take the user's emotional state into consideration, and allows for a real-time simulation of the arrangement of selected furniture in a store, making it easier to get an idea of ​​the appearance before purchasing.

[1856] "Room layout data" is spatial information including the structure, size, shape, furniture placement area, window and door positions, and the like of the room.

[1857] "Analysis" refers to the process of analyzing the received room layout data to identify the characteristics of the room.

[1858] An "interior coordination plan" is a plan for furniture placement and color schemes that is suggested based on the room layout data and emotional state.

[1859] "Emotion recognition means" is a function that analyzes the user's emotional state using facial recognition data, etc.

[1860] The "means for listing furniture" is a function for listing necessary furniture items based on the generated interior coordination plan.

[1861] The "means for searching for products at the best price" is a function for searching for the best price for listed furniture items from multiple shopping sites on the Internet.

[1862] The "means for calculating point benefits" is a function for calculating point benefits associated with purchases on a shopping site or payment platform.

[1863] The "means for generating a purchase link" is a function that generates a purchase link to a product selected by the user.

[1864] The "means for selecting an interior coordination plan based on an emotional state" is a function for selecting an optimal interior coordination plan based on emotional data acquired by the emotion recognition means.

[1865] "Means for displaying a simulation of furniture placement using AR technology" refers to a function that uses augmented reality technology to display a simulation of the listed furniture items being placed in the user's room.

[1866] This invention is a system that proposes optimal interior coordination to users based on room layout data and allows them to purchase that furniture at the optimal price. Furthermore, by combining it with an emotion engine, it is possible to recognize the user's emotional state and make suggestions based on that. To implement this invention, the following system is required.

[1867] System Configuration

[1868] The system mainly functions between a server, a terminal, and a user. The user inputs the room layout data using the terminal, and the system proposes an interior coordination plan based on the data and emotional data, and provides optimal purchasing options.

[1869] Hardware and Software Use

[1870] Hardware:

[1871] Facial Recognition Camera

[1872] Smartphone

[1873] software:

[1874] OpenAI API: Used to generate emotion-based furniture recommendations.

[1875] EmotionRecognizer: A library for emotion analysis based on facial recognition data.

[1876] ARRenderer: A renderer that displays furniture simulations using AR technology.

[1877] OpenCV: A library used to capture and display camera footage.

[1878] Furniture API: An API for retrieving furniture data and comparing prices.

[1879] Data processing and data calculation

[1880] 1. The user takes a floor plan and photos of the room and uploads them to the device.

[1881] The device receives this floor plan data and sends it to a server, which temporarily stores it and analyzes it using AI algorithms. The analysis identifies spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[1882] 2. Based on the analysis results, the server generates multiple interior coordination plans, taking into consideration the user's style, color scheme, furniture placement, and other factors.

[1883] 3. The user inputs emotional data into the device. This emotional data can be obtained by facial recognition using a webcam, text input, or voice input. Once the device receives the emotional data, it is sent to the server. The server uses an emotion engine to analyze the emotional data and identify the user's emotional state.

[1884] 4. The server selects the most appropriate interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, it will suggest furniture with calming colors and soft textures.

[1885] 5. The server generates a list of necessary furniture items based on each interior coordination plan. It then uses APIs and scraping technology to search for products at the best prices across multiple shopping sites. It retrieves and lists the lowest price information (price, store, rating, availability, etc.) found for each furniture item. It also calculates the points rewards for each item.

[1886] 6. The server sends information including the calculation results to the terminal, which displays it to the user through an interface. The user reviews the proposed interior coordination plan and furniture list and selects the one they like. The terminal resends the user's selection to the server. The server generates a purchase link for the selected furniture item and sends it to the terminal. The terminal displays the purchase link to the user and supports the purchase process.

[1887] Usage example

[1888] Users use their smartphones in physical stores to upload their room layout data. The server analyzes this data and makes suggestions. The user then uses a camera in the store to provide emotional data, allowing the server to understand the user's desire for relaxation and suggest relaxing interior designs. These interior design suggestions are simulated in real time on the smartphone using AR technology, allowing the user to view them on the spot. Next, furniture with the best price and point benefits is listed, and a purchase link is generated for the furniture selected by the user, allowing for easy purchase procedures.

[1889] Example prompts for generative AI models

[1890] Room data: {"width": 5.0, "length": 6.0, "height": 2.5, "objects": []}

[1891] Emotion: Relaxed

[1892] Suggest furniture and layout:

[1893] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1894] Step 1:

[1895] Users take floor plans and photos of their rooms and upload them to their devices.

[1896] Input: Room floor plan and photos

[1897] Output: Floor plan data saved on the device

[1898] Specific operation: Launch the camera app on your smartphone, take a photo of the floor plan or room, and upload it to the app. The uploaded data is temporarily stored on the device.

[1899] Step 2:

[1900] The terminal receives the room layout data uploaded by the user and transmits it to the server via the interface.

[1901] Input: Floor plan data saved on the device

[1902] Output: Floor plan data sent to the server

[1903] Specific operation: The application on the device sends an HTTP request to the server for the saved floor plan data. The floor plan data is attached to the request.

[1904] Step 3:

[1905] The server temporarily stores the received floor plan data and prepares it for analysis.

[1906] Input: Submitted floor plan data

[1907] Output: Floor plan data saved on the server

[1908] Specific operation: The server receives an HTTP request and temporarily stores the floor plan data in a database. After storing it, it prepares the data so that the analysis program can call it.

[1909] Step 4:

[1910] The server analyzes the stored floor plan data using AI algorithms to identify spatial characteristics such as room size, shape, available furniture placement areas, and the location of windows and doors.

[1911] Input: Floor plan data

[1912] Output: Analysis results identifying room characteristics

[1913] How it works: An AI program on the server analyzes the floor plan data and extracts various room characteristics, including the room's dimensions, wall position, and window and door placement.

[1914] Step 5:

[1915] The server generates a plurality of interior coordination plans based on the analysis results.

[1916] Input: Analysis results

[1917] Output: Interior coordination plan

[1918] How it works: The server's coordinate generator generates different designs and furniture layout plans based on the analysis results, including suggestions based on different styles and themes.

[1919] Step 6:

[1920] The user can input emotion data into the terminal by using face recognition via a webcam, text input, or voice input.

[1921] Input: Emotion data (facial recognition data, text, voice)

[1922] Output: Emotion data stored on the device

[1923] What it does: Users can use their smartphone camera to take a picture of their face so the application can analyze emotional data, or they can use text or voice input methods.

[1924] Step 7:

[1925] The terminal receives the user's emotion data and transmits it to the server.

[1926] Input: Emotion data stored on the device

[1927] Output: Emotion data sent to the server

[1928] Specific operation: The device application sends an HTTP request to the server with emotion data.

[1929] Step 8:

[1930] The server uses an emotion engine to analyze the received emotion data and identify the user's emotional state.

[1931] Input: Emotion data

[1932] Output: Emotional state analysis result

[1933] Specific operation: The emotion engine on the server analyzes the emotion data and identifies the user's emotional state (e.g., relaxed, excited, stressed, etc.).

[1934] Step 9:

[1935] The server selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine.

[1936] Input: Emotional state analysis results, interior coordination plan

[1937] Output: Optimal interior coordination plan based on emotional state

[1938] Specific operation: The server's selection program selects the interior coordination plan that best suits the user's emotional state.

[1939] Step 10:

[1940] The server generates a list of necessary furniture items based on each interior coordination plan.

[1941] Input: Interior coordination plan

[1942] Output: A list of furniture items

[1943] How it works: The server's database and algorithms create a list of the furniture items required based on the selected plan.

[1944] Step 11:

[1945] The server then begins searching for the listed furniture items using APIs and scraping technology to find the best prices on multiple online shopping sites.

[1946] Input: A list of furniture items

[1947] Output: Product information at the best price

[1948] How it works: The server's search algorithm uses each shopping site's API and scraping technology to obtain price information for the listed furniture items.

[1949] Step 12:

[1950] The server retrieves and lists the cheapest product information (price, store, rating, availability, etc.) found for each furniture item.

[1951] Input: Search results

[1952] Output: Information about the cheapest product

[1953] Specific operation: Based on the obtained price information, the server lists the cheapest products and saves them in the database.

[1954] Step 13:

[1955] Based on the search results, the server calculates the point rewards (such as points redemption on shopping sites or payment platforms) for each product.

[1956] Input: Information about the cheapest product

[1957] Output: Point reward information

[1958] Specific operation: The server calculates the point reward information for each product based on the point redemption rules of the shopping site or payment platform.

[1959] Step 14:

[1960] The server transmits the calculated reward information and the search results to the terminal.

[1961] Input: Point reward information, lowest price product information

[1962] Output: Interior coordination plan, furniture list, and point reward information sent to the terminal

[1963] Specific operation: The server sends the calculation results and point reward information together to the terminal so that the user can check them.

[1964] Step 15:

[1965] The terminal displays the interior coordination plan, furniture list, and point benefit information received from the server to the user through an interface.

[1966] Input: Interior coordination plan, furniture list, point reward information

[1967] Output: Information displayed in the interface

[1968] Specific operation: The terminal application displays the information received from the server in the user interface and allows the user to make a selection.

[1969] Step 16:

[1970] The user checks the proposed interior coordination plan and furniture list and selects the one they like.

[1971] Input: User's choice

[1972] Output: User selection information

[1973] Specific operation: The user operates the terminal interface to select the desired item from the proposed coordination plan and furniture list.

[1974] Step 17:

[1975] The terminal accepts the user's selection and retransmits the information to the server.

[1976] Input: User selection information

[1977] Output: Selections sent to the server

[1978] Specific operation: The terminal makes an HTTP request again to send the user's selection information to the server.

[1979] Step 18:

[1980] The server generates a purchase link for the furniture item selected by the user and sends it to the terminal.

[1981] Input: User selection information

[1982] Output: Purchase link

[1983] Specific operation: The server generates a purchase link for the specified furniture item and sends it to the device.

[1984] Step 19:

[1985] The terminal displays a purchase link to the user and assists in the purchase procedure.

[1986] Enter: Purchase Link

[1987] Output: Display link to user

[1988] What happens: The device displays a purchase link in the user interface and helps the user click the link to actually complete the purchase.

[1989] Step 20:

[1990] Users use the provided purchase link to purchase the furniture through a shopping site or payment platform.

[1991] Enter: Purchase Link

[1992] Output: Purchase complete

[1993] What happens: The user clicks on the purchase link, is taken to the shopping site, and completes the purchase.

[1994] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1995] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1996] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1997] [Fourth embodiment]

[1998] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1999] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2000] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2001] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2002] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2003] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2004] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2005] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2006] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2007] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2008] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2009] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2010] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2011] This invention relates to a system that proposes optimal interior coordination to users based on room layout data and also allows them to purchase the furniture at the optimal price. The specific operations performed in the processing of each step are described below.

[2012] System Configuration

[2013] The system mainly works between a server, a terminal, and a user. The user inputs the layout data of the room using the terminal, and the server uses that data to suggest interior coordination and provide optimal purchasing options.

[2014] Program processing

[2015] 1. Enter and submit room layout data

[2016] User: Takes a floor plan and photographs of the room, and uploads them to the system via a device.

[2017] Terminal: Sends the uploaded data to the server.

[2018] Server: Temporarily stores received data and prepares it for analysis.

[2019] 2. Analysis of room layout data

[2020] Server: Analyzes the received floor plan data using an AI algorithm to identify spatial characteristics such as room size, shape, available furniture placement areas, and the location of windows and doors.

[2021] Example: If a floor plan is divided into a living room, bedroom, and kitchen, the system automatically recognizes the area and shape of each room, as well as the space available for furniture placement.

[2022] 3. Creating an interior coordination plan

[2023] Server: Based on the analysis results, it generates multiple interior coordination plans according to the purpose of use of the space and the user's preferences.

[2024] Example: For a living room, suggestions include a Scandinavian-style furniture arrangement plan and a modern furniture arrangement plan.

[2025] 4. List your furniture items

[2026] Server: Generates a list of required furniture items based on the proposed interior coordination plan.

[2027] Example: Select key items for each category, such as sofas, dining tables, and beds.

[2028] 5. Find the best priced furniture

[2029] Server: Searches multiple online shopping sites for the best price for the listed furniture items.

[2030] Example: Search for "L-shaped sofa" on multiple shopping sites and determine the best product based on price, reviews, delivery terms, etc.

[2031] 6. Calculation of points rewards

[2032] Server: Calculates the points reward for each product based on the search results.

[2033] Example: Calculate the reward points when purchasing using a combination of Yahoo! Shopping and PayPay.

[2034] 7. Providing Information to the User Interface

[2035] Server: Sends calculated reward information and search results to the device.

[2036] Terminal: Displays proposed interior plans and product lists to the user through a user interface.

[2037] 8. Generate Purchase Link

[2038] User: Select the best from the proposed interior plans and product list.

[2039] Server: Generates a purchase link for the product selected by the user and sends it to the device.

[2040] Device: Show the purchase link to the user and assist them in the purchase process.

[2041] Specific examples

[2042] A user wants to redecorate their living room, so they upload a floor plan. The server analyzes the living room, recognizes its size and shape, and suggests furniture placement. It then searches the internet for the best prices for each furniture item and calculates the combination that maximizes point rewards. Finally, it generates a purchase link and presents it to the user, allowing them to easily purchase the furniture.

[2043] This system allows users to effortlessly coordinate their interiors and purchase furniture economically all in one place.

[2044] The processing flow will be explained below.

[2045] Step 1:

[2046] User: Take a floor plan or photo of the room and upload it to the device.

[2047] Step 2:

[2048] Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[2049] Step 3:

[2050] Server: Temporarily stores the received floor plan data and prepares it for data analysis.

[2051] Step 4:

[2052] Server: Analyzes stored floor plan data using AI algorithms to identify spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[2053] Step 5:

[2054] Server: Generates multiple interior coordination plans based on the analysis results, proposing plans that take into account style, color scheme, furniture placement, etc.

[2055] Step 6:

[2056] Server: Generates a list of required furniture items based on each interior coordination plan.

[2057] Step 7:

[2058] Server: Initiates a search for the listed furniture items using APIs and scraping technology to find the best prices from multiple shopping sites.

[2059] Step 8:

[2060] Server: Retrieves and lists the cheapest price information (price, retailer, ratings, availability, etc.) found for each furniture item.

[2061] Step 9:

[2062] Server: Based on the search results, calculate the points rewards for each product (such as Yahoo! Shopping or PayPay points).

[2063] Step 10:

[2064] Server: Sends the interior coordination plan and furniture list including the calculation results to the terminal.

[2065] Step 11:

[2066] Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through an interface.

[2067] Step 12:

[2068] User: Review the proposed interior coordination plan and furniture list and select the one they like.

[2069] Step 13:

[2070] Terminal: Accepts the user's selection and resends the information to the server.

[2071] Step 14:

[2072] Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[2073] Step 15:

[2074] Device: Show the purchase link to the user and assist them in the purchase process.

[2075] Step 16:

[2076] User: Use the provided purchase link to purchase furniture through a shopping site or PayPay.

[2077] Step 17:

[2078] Server: Follow up with the user after the purchase by sending them a confirmation of the purchase and tracking information.

[2079] Example 1

[2080] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2081] Conventional interior coordination systems lacked the functionality to suggest optimal furniture placement to users. Furthermore, providing information on how to purchase furniture at the best price and calculating point rewards associated with purchases were often done manually, which was time-consuming for users. Furthermore, the systems lacked the means to safely handle users' personal information, leaving them at risk of data theft and unauthorized access.

[2082] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2083] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching multiple online shopping sites for products at the best price for the listed furniture, means for calculating points related to purchases from the search results, means for providing the user with information including the calculation results, means for generating a purchase link to the product selected by the user, and means for encrypting and transmitting data to protect the user's privacy. This allows the user to effortlessly coordinate an optimal interior and purchase furniture economically all in one place.

[2084] "Room layout data" refers to data provided by users that includes information such as the dimensions and shape of the room, the area in which furniture can be placed, and the positions of windows and doors.

[2085] The "receiving means" is a system or device for receiving room layout data from a user.

[2086] "Means for analysis" refers to a system or device that analyzes the received room layout data using an AI algorithm or the like to identify the structure and characteristics of the room.

[2087] A "means for identifying features" is a system or device that has the ability to extract important spatial information from the analysis results, such as the size and shape of the room, the area where furniture can be placed, and the location of windows and doors.

[2088] An "interior coordination plan" is data that shows furniture placement and design plans proposed based on the user's preferences and room characteristics.

[2089] The "generating means" is a system or device for creating a plurality of interior coordination plans based on the analysis results.

[2090] The "listing means" is a system or device having a function for creating a list of necessary furniture and decorative items based on the generated interior coordination plan.

[2091] The "means for searching for products at the best price" is a system or device that has the function of searching for the most favorable priced product for the listed furniture from multiple online shopping sites.

[2092] The "means for calculating point benefits" is a system or device that has the function of calculating the total points redeemed based on the point redemption information provided by multiple shopping sites on the Internet based on search results.

[2093] The "means for providing information including calculation results to users" refers to a system or device for notifying users of search results including point redemption information and visually displaying them.

[2094] The "means for generating a purchase link" refers to a system or device for creating a link that allows a user to directly purchase the product selected by the user on an online shopping site.

[2095] "Means for encryption and transmission" refers to a system or device that encrypts data such as floor plan data and user information and communicates them securely in order to protect user privacy.

[2096] This invention relates to a system that proposes optimal interior coordination to users based on room layout data and allows them to purchase the furniture at the optimal price. This system functions in cooperation with a server, terminals, and users.

[2097] System Configuration

[2098] The user inputs the layout data of the room using a terminal, and the server uses that data to propose an interior coordination plan and also provides optimal purchasing options. In terms of specific hardware, the user's terminal can be a smartphone, tablet, or PC, while the server uses a high-performance computer system.

[2099] Entering and Submitting Data

[2100] User

[2101] Users take photos of the room floor plan and other images using the camera on their smartphone or PC, and upload the data to the system using the following prompts:

[2102] "I uploaded the living room floor plan."

[2103] "I sent you the bedroom floor plan."

[2104] Terminal

[2105] The terminal temporarily stores the data entered by the user and transmits it to the server using a secure communication protocol (HTTPS).

[2106] server

[2107] The server temporarily stores the received data and checks the data for consistency and format. If any errors are found, the user is notified and asked to re-upload.

[2108] Data analysis

[2109] server

[2110] The server uses AI algorithms (e.g., Convolutional Neural Networks) to analyze the data it receives, identifying important spatial features such as the room's size, shape, available furniture area, and the location of windows and doors.

[2111] As a specific example, if a floor plan is divided into a living room, bedroom, and kitchen, the area and shape of each room, as well as the space available for furniture placement, are automatically recognized using libraries such as HTML5 Canvas and OpenCV.

[2112] Coordination plan generation

[2113] server

[2114] The server generates multiple interior coordination plans based on the above analysis results, tailored to the user's preferences and lifestyle. Using a specific generation AI model, the server generates plans using the following prompts:

[2115] "Please suggest a Scandinavian-style interior design for the living room."

[2116] "Generate a modern bedroom furniture arrangement plan."

[2117] The generated plans are offered for selection by the user.

[2118] Listing furniture items

[2119] server

[2120] Based on the plan generated, create a list of the furniture items you'll need, including items in each major category, such as a sofa, dining table, and bed.

[2121] Find the best price

[2122] server

[2123] The server searches multiple online shopping sites in real time for the best price for the listed furniture, taking into account not only price but also ratings, delivery terms, and other factors.

[2124] Points Reward Calculation

[2125] server

[2126] Based on the search results, the points rewards for each product are calculated. For example, the total points are calculated based on the point reward rates on Yahoo! Shopping and other online platforms.

[2127] Providing information

[2128] server

[2129] Information including the calculation results is sent to the terminal, which then visually displays it so that the user can easily understand it.

[2130] Generate purchase links

[2131] User

[2132] The user selects the most suitable item from the proposed interior plans and product list.

[2133] server

[2134] The server generates a purchase link for the product selected by the user and sends it to the terminal, which displays the link to the user and assists in the purchase process.

[2135] Data encryption

[2136] Terminals and Servers

[2137] To protect user privacy, all data transmitted is encrypted using Transport Layer Security (TLS), which protects user information from data theft and unauthorized access.

[2138] As described above, this system minimizes the user's effort and enables optimal interior coordination and economical furniture purchases.

[2139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2140] Step 1:

[2141] input

[2142] User: Take a floor plan or photo of a room.

[2143] Users: Upload data via a dedicated application or web portal.

[2144] output

[2145] Device: Temporarily stores floor plans and photo data.

[2146] Terminal: Sends data to the server using a secure communication protocol (HTTPS).

[2147] Specific actions

[2148] Users take a picture of the room floor plan using the camera on their smartphone or PC.

[2149] The captured data is saved in the application and sent to the server.

[2150] Step 2:

[2151] input

[2152] Terminal: Receives floor plan data from the user.

[2153] Server: Receives floor plan data sent from the device.

[2154] output

[2155] Server: Temporarily stores the received data and checks its structural integrity.

[2156] Server: If the data is in the correct format, prepare it for analysis. If it is not, send an error notification to the user.

[2157] Specific actions

[2158] The server checks the integrity of the data and notifies the user if there are any discrepancies.

[2159] If the data format is correct, proceed to the next analysis step.

[2160] Step 3:

[2161] input

[2162] Server: Received floor plan data

[2163] output

[2164] Server: Analyzes floor plan data and identifies room size, shape, available furniture placement area, and window and door locations.

[2165] Specific actions

[2166] The server analyzes the floor plan data using an AI algorithm (e.g., Convolutional Neural Network).

[2167] Identify room boundaries, dimensions, and functional areas and store them in a database.

[2168] Step 4:

[2169] input

[2170] Server: Analysis results (room size, shape, furniture placement area, etc.)

[2171] output

[2172] Server: Multiple interior coordination plans based on user preferences

[2173] Specific actions

[2174] The server uses a generative AI model to generate a coordination plan based on the prompt sentence.

[2175] Example: "Generate a Scandinavian furniture arrangement plan for my living room."

[2176] Organize the proposed plans and select the most suitable one.

[2177] Step 5:

[2178] input

[2179] Server: Generated interior coordination plan

[2180] output

[2181] Server: List of required furniture items

[2182] Specific actions

[2183] The server lists the necessary furniture items based on the coordination plan.

[2184] Select key items for each category, such as sofas and dining tables.

[2185] Step 6:

[2186] input

[2187] Server: List of furniture items

[2188] output

[2189] Server: A list of products with the best prices from multiple shopping sites on the Internet

[2190] Specific actions

[2191] The server crawls multiple shopping sites and compares prices.

[2192] Example: Search for "L-shaped sofa" across multiple sites to identify the best products.

[2193] Product ratings and delivery terms are also taken into consideration.

[2194] Step 7:

[2195] input

[2196] Server: Price comparison results

[2197] output

[2198] Server: Calculation result of point benefits for each product

[2199] Specific actions

[2200] The server calculates the point rewards for each product based on the point redemption rate.

[2201] Example: Calculating Yahoo! Shopping reward points.

[2202] Step 8:

[2203] input

[2204] Server: Calculated point reward information

[2205] Terminal: Information received from the server

[2206] output

[2207] Terminal: Displays interior plans and product lists in a user-friendly format

[2208] Specific actions

[2209] The terminal retrieves the calculated reward information and displays it in a graphical interface.

[2210] Product images, price information, point redemption information, and more can be viewed at a glance.

[2211] Step 9:

[2212] input

[2213] User: Select the furniture item they want from the list of suggested items

[2214] output

[2215] Server: Purchase link for selected product

[2216] Specific actions

[2217] The server generates a purchase link for the furniture item selected by the user.

[2218] Send a purchase link to the device to allow users to easily make a purchase.

[2219] By following the above steps, users can efficiently and easily coordinate their interiors optimally and purchase furniture economically.

[2220] (Application example 1)

[2221] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2222] Currently, when coordinating interiors, it is difficult to select and arrange the optimal furniture to fit the floor plan. Furthermore, there is a lack of ways to purchase furniture at the best price or to virtually check what the interior will look like after purchase. This requires users to spend a lot of time and effort, making it difficult to achieve effective interior coordination.

[2223] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2224] In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for listing furniture based on the generated interior coordination plans, means for searching multiple online shopping sites for products at the best prices for the listed furniture, means for calculating point benefits associated with purchases from the search results, means for providing the user with information including the calculation results, means for generating purchase links to products selected by the user, means for the user to 3D scan the floor plan data, and means for confirming the interior coordination plan in a virtual space using the 3D scanned data. This allows the user to efficiently and economically realize optimal interior coordination and virtually confirm it.

[2225] "Room layout data" is information that indicates the layout, size, and shape of a physical space such as a user's residence or office.

[2226] "Means for receiving" refers to devices or software that have the functionality to allow the system to acquire data sent from the user.

[2227] "Means for analysis" refers to algorithms or software that analyze the received data and identify the structure and characteristics of the space.

[2228] An "interior coordination plan" is a specific proposal for the layout and design of furniture and decorations to be placed in a room.

[2229] The "listing means" refers to a device or software that has the function of selecting the necessary furniture and products based on the interior coordination plan.

[2230] "Internet shopping site" refers to a website where users can search for and purchase products online.

[2231] A "means for searching for products at the best price" is an algorithm or software that searches multiple shopping sites for products that meet your criteria.

[2232] The "means for calculating point benefits" refers to a device or software that has the function of calculating points or discounts that can be obtained at the time of purchase.

[2233] "Means for providing information to users" refers to the interface or software for displaying calculation results and proposals to users.

[2234] The "means for generating a purchase link" refers to a device or software that has the function of creating a link for purchasing the product selected by the user.

[2235] "3D scanning means" refers to devices and software for digitizing physical spaces as three-dimensional data.

[2236] "Means for checking in a virtual space" refers to devices or software that have the function of allowing users to visually check interior coordination plans using three-dimensional models.

[2237] The present invention relates to a system that allows a user to propose optimal interior coordination based on room layout data, and detailed embodiments thereof will be described.

[2238] System Configuration

[2239] The system mainly functions between the server, the terminal, and the user. The system receives room layout data and 3D scan data from the user and proposes interior coordination in the virtual space. Specifically, it works as follows:

[2240] Hardware and Software

[2241] Hardware:

[2242] A smartphone or head-mounted display (e.g., Oculus Quest 2)

[2243] Server (Cloud server or on-premise server)

[2244] software:

[2245] Unity, Vuforia (AR technology), OpenCV

[2246] Python and Flask (server-side processing)

[2247] MongoDB (data storage)

[2248] Processing flow

[2249] Data reception and analysis

[2250] 1. User:

[2251] Use a smartphone or head-mounted display to 3D scan the layout of a room.

[2252] The acquired data is sent from the terminal to the server.

[2253] 2. Server:

[2254] The server analyzes the received 3D scan data using OpenCV to determine the size, shape, and area where furniture can be placed in the room.

[2255] The analysis results are saved in MongoDB.

[2256] Generate interior coordination plans

[2257] 3. Server:

[2258] Based on the analysis results, a Python script is used to generate multiple interior coordination plans, such as Scandinavian and modern styles.

[2259] Check in virtual space

[2260] 4. User:

[2261] Using Unity and Vuforia, participants can visually check 3D models of proposed interior coordination plans in a virtual space using a head-mounted display or smartphone.

[2262] Product listing and price search

[2263] 5. Server:

[2264] The system creates a list of furniture items based on the generated interior coordination plan and uses a Python script to search multiple online shopping sites for the best prices.

[2265] Calculate points rewards for each product based on search results.

[2266] Purchase links and information

[2267] 6. Server:

[2268] The search results and calculated point reward information are sent to the terminal and displayed to the user.

[2269] A purchase link is generated for the product selected by the user and sent to the terminal.

[2270] 7. Users:

[2271] Check the proposed interior plans and product list and select the most suitable products.

[2272] Easily purchase products through our purchase links.

[2273] Prompt Sentence Examples

[2274] The invention provides a virtual interior coordination assistant application that allows users to send scanned data of a room to a server and easily coordinate the optimal interior and compare prices.

[2275] As a concrete example, a user considers redecorating their bedroom and scans the room using a head-mounted display. The system analyzes the size and shape of the room and proposes interior plans in both Scandinavian and modern styles. The user can then view the virtual 3D models and purchase furniture that suits their tastes at the best price.

[2276] This system allows users to efficiently coordinate their interiors optimally and purchase furniture at economical prices.

[2277] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2278] Step 1:

[2279] Users use a smartphone or head-mounted display to 3D scan the layout of a room, and the scanned data is then sent from the device to a server.

[2280] Input: 3D scan of a room.

[2281] Processing: The device receives the 3D scan data taken by the user, converts it into the appropriate format, and sends it to the server.

[2282] Output: 3D scan data sent to the server.

[2283] Step 2:

[2284] The server analyzes the received 3D scan data to determine the size, shape, and available furniture area of ​​the room using OpenCV.

[2285] Input: 3D scan data.

[2286] Processing: Using OpenCV, the server analyzes the 3D scan data and extracts spatial features (size, shape, furniture placement area), applying image processing techniques such as edge detection and shape analysis.

[2287] Output: Data about the room size, shape, and available furniture area.

[2288] Step 3:

[2289] The server generates multiple interior coordination plans based on the analysis results, using a Python script.

[2290] Input: Data about the room size, shape, and available furniture area.

[2291] Processing: Using a Python script, multiple interior coordination plans (e.g., Scandinavian style, modern style) are generated based on the analysis results. Various plans are created by referencing past design data and user preferences.

[2292] Output: Interior coordination plan data.

[2293] Step 4:

[2294] Using Unity and Vuforia, users can visually view 3D models of proposed interior coordination plans in a virtual space using a head-mounted display or smartphone.

[2295] Input: Interior coordination plan data.

[2296] Processing: Using Unity and Vuforia, the interior coordination plan is created as a 3D model and displayed in a user interface, allowing users to visually inspect the model and perform operations such as rotating and scaling.

[2297] Output: A virtual interior coordination that can be visually confirmed by the user.

[2298] Step 5:

[2299] The server lists furniture items based on the interior coordination plan and uses a Python script to search multiple shopping sites on the Internet for the best priced items.

[2300] Input: Interior coordination plan data.

[2301] Processing: The server crawls the web for the listed furniture items, collects information from multiple shopping sites, and compares and selects the best options based on price, reviews, delivery terms, etc.

[2302] Output: A list of the best purchasing options for each furniture item.

[2303] Step 6:

[2304] The server calculates the points reward for each product based on the search results.

[2305] Enter: the best buying options list.

[2306] Processing: The server obtains the point reward information of each shopping site and calculates the number of points that can be earned at the time of purchase. This is done to calculate the economic efficiency of the purchase.

[2307] Output: A list of optimal purchase options with point rewards.

[2308] Step 7:

[2309] The server generates the calculation results and a link to purchase the furniture, and sends them to the device. The user can then use their smartphone or head-mounted display to check the interior plan and product list and select the most suitable product. The product can then be easily purchased using the purchase link.

[2310] Enter: a list of the best purchasing options, including points rewards.

[2311] Processing: Organize the calculated reward information and product list, generate data to display in the user interface, generate a purchase link for the product selected by the user, and send it to the device.

[2312] Output: Interior plans and product list with purchase links.

[2313] This series of processes allows users to efficiently and economically create optimal interior coordination and easily purchase furniture while checking it virtually.

[2314] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2315] This invention combines an emotion engine with a system that proposes optimal interior coordination to users based on room layout data and then allows them to purchase that furniture at the optimal price. The emotion engine recognizes the user's emotional state and makes suggestions based on that.

[2316] System Configuration

[2317] The system mainly works between the server, the terminal, and the user. The user inputs the room layout data using the terminal, and the server proposes interior coordination based on that data and the user's emotional data, and provides optimal purchasing options.

[2318] Program processing

[2319] Enter and submit room layout data

[2320] 1. User: Take a floor plan or photo of the room and upload it to the device.

[2321] 2. Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[2322] 3. Server: Temporarily stores the received data and prepares it for analysis.

[2323] Analysis of room layout data

[2324] 4. Server: The stored floor plan data is analyzed using AI algorithms to identify spatial characteristics such as room size, shape, furniture placement area, and window and door locations.

[2325] Generate interior coordination plans

[2326] 5. Server: Based on the analysis results, it generates multiple interior coordination plans, taking into account style, color scheme, furniture placement, etc.

[2327] Emotion data input and analysis

[2328] 6. User: Emotional data can be input to the device using facial recognition via a webcam, text input, or voice input.

[2329] 7. Terminal: Receives the user's emotion data and sends it to the server.

[2330] 8. Server: Analyzes the received emotion data using the emotion engine and identifies the user's emotional state.

[2331] Generate personalized proposals

[2332] 9. Server: Selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, it will suggest furniture with calming colors and soft textures.

[2333] Listing furniture items

[2334] 10. Server: Generates a list of required furniture items based on each interior coordination plan.

[2335] Find the best priced furniture

[2336] 11. Server: Initiates a search for the listed furniture items using APIs and scraping techniques to find the best prices from multiple shopping sites.

[2337] 12. Server: Retrieves and lists the cheapest price information (price, retailer, rating, availability, etc.) found for each furniture item.

[2338] Points Reward Calculation

[2339] 13. Server: Based on the search results, calculates the points rewards for each product (such as points redemption on shopping sites or payment platforms).

[2340] Providing information to the user interface

[2341] 14. Server: Sends the calculated reward information and search results to the device.

[2342] 15. Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through the interface.

[2343] Generate purchase links

[2344] 16. User: Review the proposed interior coordination plan and furniture list and select the one they like best.

[2345] 17. Terminal: Accepts the user's selection and resends the information to the server.

[2346] 18. Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[2347] 19. Device: Display the purchase link to the user and assist with the purchase process.

[2348] Purchasing and Follow-up

[2349] 20. User: Uses the provided purchase link to purchase furniture through a shopping site or payment platform.

[2350] 21. Server: Follow up on the purchase by sending the user a confirmation of the purchase and tracking information.

[2351] Specific examples

[2352] A user wants to redecorate their living room and uploads a floor plan. The server analyzes the living room, recognizes its size and shape, and suggests furniture placement. The user then provides emotional data via webcam, revealing that they are looking for a relaxing space. Based on this information, the server suggests furniture with calming color schemes and interior design that creates a relaxing effect. The server searches multiple shopping sites for the best prices and point rewards for the furniture and provides them to the user. Once the user selects the furniture, a purchase link is generated to help them easily complete the purchase process.

[2353] This system allows users to coordinate their interiors to best suit their emotional state and makes hassle-free and economical furniture purchases all in one go.

[2354] The processing flow will be explained below.

[2355] Step 1:

[2356] User: Take a floor plan or photo of the room and upload it to the device.

[2357] Step 2:

[2358] Terminal: Receives the room layout data uploaded by the user and sends it to the server through the interface.

[2359] Step 3:

[2360] Server: Temporarily stores the received floor plan data and prepares it for data analysis.

[2361] Step 4:

[2362] Server: Analyzes stored floor plan data using AI algorithms to identify spatial features such as room size, shape, available furniture placement areas, and window and door locations.

[2363] Step 5:

[2364] Server: Generates multiple interior coordination plans based on the analysis results, proposing plans that take into account style, color scheme, furniture placement, etc.

[2365] Step 6:

[2366] User: Enters emotion data into the device using face recognition via a webcam, text input, or voice input.

[2367] Step 7:

[2368] Terminal: Receives the user's emotion data and sends the emotion data to the server.

[2369] Step 8:

[2370] Server: Analyzes the received emotional data using the emotion engine and identifies the user's emotional state. If the user wants to relax, it will be labeled as "Relaxed," and if the user wants to be energized, it will be labeled as "Energy."

[2371] Step 9:

[2372] Server: Selects an interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking for relaxation, the server will suggest furniture with calming colors and soft textures.

[2373] Step 10:

[2374] Server: Generates a list of required furniture items based on each interior coordination plan.

[2375] Step 11:

[2376] Server: Initiates a search for the listed furniture items using APIs and scraping technology to find the best prices from multiple shopping sites.

[2377] Step 12:

[2378] Server: Retrieves and lists the cheapest price information (price, retailer, ratings, availability, etc.) found for each furniture item.

[2379] Step 13:

[2380] Server: Based on the search results, calculates the reward points for each product (such as points redemption from shopping sites or payment platforms).

[2381] Step 14:

[2382] Server: Sends the interior coordination plan and furniture list including the calculation results to the terminal.

[2383] Step 15:

[2384] Terminal: The interior coordination plan, furniture list, and point benefit information received from the server are displayed to the user through an interface.

[2385] Step 16:

[2386] User: Review the proposed interior coordination plan and furniture list and select the one they like.

[2387] Step 17:

[2388] Terminal: Accepts the user's selection and resends the information to the server.

[2389] Step 18:

[2390] Server: Generates a purchase link for the furniture item selected by the user and sends it to the device.

[2391] Step 19:

[2392] Device: Show the purchase link to the user and assist them in the purchase process.

[2393] Step 20:

[2394] User: Purchases furniture through a shopping site or payment platform using the provided purchase link.

[2395] Step 21:

[2396] Server: Follow up with the user after the purchase by sending them a confirmation of the purchase and tracking information.

[2397] Example 2

[2398] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2399] Conventional interior coordination suggestion systems have difficulty making personalized suggestions that take into account the user's emotional state. Furthermore, searching for furniture prices and presenting point benefits is time-consuming, and the system provides insufficient support for users to make optimal choices. This makes it difficult to achieve interior coordination that satisfies users and to efficiently purchase furniture.

[2400] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving room layout data, means for analyzing the received floor plan data and identifying the structure and characteristics of the room, means for generating multiple interior coordination plans based on the analysis results, means for acquiring and analyzing the user's emotional state, means for selecting an appropriate interior coordination plan based on the acquired emotional state, means for listing furniture based on the generated interior coordination plan, means for searching multiple shopping sites on the Internet for products at the best prices for the listed furniture, means for calculating point benefits related to purchases from the search results, means for providing the user with information including the calculation results, and means for generating purchase links to products selected by the user. This allows the server to suggest optimal interior coordination based on the user's emotional state, enabling efficient and economical furniture purchases.

[2401] "Room layout data" is numerical and graphical information that indicates the shape and dimensions of a room, the area in which furniture can be placed, the positions of windows and doors, and so on.

[2402] An "interior coordination plan" is a proposal for the interior design of a room, including the style, color scheme, furniture placement, and decorative items.

[2403] "Emotional state" is information that indicates the user's mood and state of mind, and is primarily obtained through facial recognition, voice analysis, text input, etc.

[2404] A "furniture list" is a list of necessary furniture items based on a specific interior coordination plan, including detailed information about each item.

[2405] The "best price" is the lowest price obtained from multiple shopping sites on the Internet for the furniture item specified by the user.

[2406] "Point benefits" are points or discount information provided by shopping sites or payment platforms that are given back to users when they make a purchase.

[2407] A "purchase link" is a direct URL or hyperlink to an online shopping site that allows a user to purchase a specified furniture item.

[2408] This invention is a system that proposes optimal interior coordination to users based on room layout data. This system makes personalized suggestions based on the user's emotional state, and also allows the user to purchase furniture at the optimal price based on those suggestions.

[2409] System Configuration

[2410] This system mainly functions between a server, a terminal, and a user. The user inputs room layout data using the terminal, and the server proposes interior coordination based on that data and the user's emotional data, and provides optimal purchasing options.

[2411] Hardware and software used

[2412] Devices: Smartphone, tablet, PC (with webcam)

[2413] Server: Cloud server (e.g. Amazon Web Services)

[2414] AI algorithm: TensorFlow (for image analysis)

[2415] Emotion analysis engine: Emotion API (face recognition, voice analysis)

[2416] Database: SQL database (for managing customer information and furniture lists)

[2417] API: Web scraping tool (for product price search)

[2418] Program processing

[2419] Enter and submit room layout data

[2420] The user uses the device to take floor plans and photos of the room and uploads them to the device through a dedicated application. The device receives the data uploaded by the user and sends it to the server.

[2421] Analysis of room layout data

[2422] The server analyzes the received floor plan data using an AI algorithm (e.g., TensorFlow) to identify spatial characteristics such as room size, shape, and available furniture placement areas, as well as the location of windows and doors.

[2423] Generate interior coordination plans

[2424] The server generates multiple interior coordination plans based on the analysis results, including different styles, color schemes, and furniture placements.

[2425] Emotion data input and analysis

[2426] Users can input emotion data into their devices using facial recognition via a webcam, text input, or voice input. The device receives the user's emotion data and sends it to a server. The server then uses an emotion analysis engine (e.g., Emotion API) to identify the user's emotional state.

[2427] Generate personalized proposals

[2428] The server selects the most appropriate interior coordination plan based on the user's emotional state analyzed by the emotion engine. For example, if the user is looking to relax, it will suggest furniture with a calm color scheme and soft textures.

[2429] Listing furniture items

[2430] The server generates a list of necessary furniture items based on the selected interior plan, including detailed information about each piece (size, design, material, etc.).

[2431] Find the best priced furniture

[2432] The server searches for the best priced products for the listed furniture items from multiple shopping sites using APIs and scraping technology, and obtains the best prices and ratings for the furniture based on the search results.

[2433] Points Reward Calculation

[2434] The server calculates the points rewards (points redeemed by shopping sites and payment platforms) for each product, allowing users to make the most economical overall choice.

[2435] Providing information to the user interface

[2436] The server transmits the calculated reward information and search results to the terminal, which displays the information to the user through an interface.

[2437] Generate purchase links

[2438] The user checks the presented interior coordination plan and furniture list and selects the furniture they want. The terminal retransmits the user's selection to the server. The server generates a purchase link for the furniture item selected by the user and transmits it again to the terminal. The terminal displays the purchase link to the user and supports the purchase procedure.

[2439] Purchasing and Follow-up

[2440] The user uses the provided purchase link to purchase the furniture through a shopping site or payment platform, and the server then sends the user a confirmation of the purchase and tracking information as follow-up after the purchase.

[2441] Specific operation example

[2442] For example, if a user is thinking about redecorating their living room, they first upload a floor plan. The server analyzes the floor plan and recognizes the size and shape of the living room. Next, the user provides emotional data via webcam, and it is determined that they are looking for a relaxing space. Based on this information, the server suggests calming color schemes and interior design that has a relaxing effect. It also searches multiple shopping sites for the best prices and point rewards for furniture and provides them to the user. The server then generates a purchase link for the furniture selected by the user, facilitating a simple purchasing process.

[2443] Prompt Sentence Examples

[2444] I'm thinking of redecorating my living room. Please suggest the best interior design based on this floor plan. I want to create a relaxing space.

[2445] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2446] Step 1:

[2447] User: Takes a photo or a floor plan of the room and uploads it to the device via a dedicated application. The request includes the floor plan and photo data. For example, a user takes a photo of the living room floor plan with their smartphone and presses the "upload" button.

[2448] Terminal: Receives floor plan data uploaded by the user, checks the data format (JPEG or PNG), and sends it to the server. Once the data format check is complete, it is sent to the server.

[2449] Specific operation: When the user presses the "Upload floor plan" button, the image data of the floor plan is saved on the device and transferred to the server.

[2450] Step 2:

[2451] Server: The received floor plan data is temporarily stored in cloud storage (e.g., Amazon S3), and then an AI algorithm (TensorFlow) is invoked to analyze it.

[2452] Input: Received image data of floor plans and photos

[2453] Output: Analytical data including room size, shape, available furniture area, and spatial characteristics such as window and door locations

[2454] Specific operation: The server passes the stored floor plan to an AI algorithm, which performs edge detection and object recognition. The analysis results are then stored in a database.

[2455] Step 3:

[2456] Server: Based on the analysis results, the server executes the interior coordination plan generation program. Here, it references several interior style templates from the database and selects a plan that suits the user's room.

[2457] Input: Room analysis data

[2458] Output: Multiple interior coordination plans (style, color scheme, furniture placement)

[2459] Specific operation: Based on the analysis data, multiple optimal templates are extracted and images and text information for presentations are generated.

[2460] Step 4:

[2461] User: Activate the device's webcam and use the facial recognition system to input emotion data, or choose to input text or voice.

[2462] Terminal: Receives the user's emotional data (facial images and voice data), sends the data to the Emotion API for real-time analysis.

[2463] Input: User emotion data (image, voice, text)

[2464] Output: User's emotional state (relaxed, stressed, excited, etc.)

[2465] What it does: The webcam captures the user's face and sends the data in real time to the Emotion API to analyze their emotional state.

[2466] Step 5:

[2467] Server: Based on the analysis results of the emotion engine, the server selects the most suitable interior coordination plan. For example, if the user is looking for relaxation, the server will suggest a calm color scheme and soft furniture.

[2468] Input: Emotional state data and initial analysis data

[2469] Output: Optimal interior coordination plan based on emotional state

[2470] Specific operation: The system automatically adjusts the priority of interior plans based on emotional data, and selects and displays the top plans.

[2471] Step 6:

[2472] Server: Generates a list of furniture items based on the selected interior coordination plan. The list includes detailed information about the items (size, design, material, etc.).

[2473] Input: Interior coordination plan

[2474] Output: A list of furniture items

[2475] Specific operation: Extract relevant furniture information from the database and compile it in list form.

[2476] Step 7:

[2477] Server: For the listed furniture items, search for the best price from multiple shopping sites using APIs and scraping technology. Collect price information and ratings.

[2478] Input: List of furniture items

[2479] Output: Optimal price and rating information for each furniture item

[2480] Specific operation: Use a web scraping tool or API to search for and collect information on relevant items on each shopping site.

[2481] Step 8:

[2482] Server: Calculates the points rewards for each product. Calculates the actual price based on the point reward information from the shopping site or payment platform.

[2483] Input: Furniture item price and special offer information

[2484] Output: Details of point redemption as calculation result

[2485] Specific operation: Calculates the point redemption rate and displays the actual payment amount and redemption points.

[2486] Step 9:

[2487] Server: Sends calculation results and analysis data to the terminal, allowing the ...

Claims

1. means for receiving room layout data; A means for analyzing the received floor plan data and identifying the structure and characteristics of the room; A means for generating a plurality of interior coordination plans based on the analysis results; A means for listing furniture based on the generated interior coordination plan; For the listed furniture, there is a way to search for the best price on multiple shopping sites on the Internet, a means for calculating point rewards associated with purchases from search results; means for providing information including the calculation results to a user; a means for generating a purchase link to the user's selected product; A system including:

2. 2. The system according to claim 1, further comprising means for generating an optimal furniture arrangement plan corresponding to an interior coordination plan selected by a user.

3. 10. The system of claim 1, further comprising means for supporting payment for the items selected by the user.

Images