System

Abstract

A system is provided.SOLUTION: A system comprising means for collecting image data based on historical examples, means for tagging specific features to the collected image data, means for receiving a generative request based on a user-suggested element, means for generating a new image based on the received generative request, means for receiving feedback from a user for evaluating the generated image, and means for updating an AI model based on the feedback.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the proposal process in the office furniture industry, it is common to use 2D floor plans, product images, and past examples to make proposals. However, these methods present challenges in that it is difficult for customers to accurately understand the proposed content. In particular, using high-resolution 3D image perspectives increases technical reproducibility and makes proposals easier for customers to understand, but creating them is time-consuming and costly. To eliminate these pain points, a new method is needed to generate high-quality proposal images efficiently and at low cost. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for collecting image data based on past cases, a means for tagging the collected image data with specific features, a means for receiving a generation request based on elements suggested by a user, a means for generating a new image based on the received generation request, a means for receiving feedback from the user for evaluating the generated image, and a means for updating an AI model based on the feedback. This system speeds up the recommendation process and reduces costs, enabling efficient generation of high-quality suggested images that meet the user's wishes.

[0006] "Past cases" refers to materials such as photographs and documents relating to previous office furniture proposals and installations.

[0007] "Image data" is digital data containing visual information, including photographs and drawings showing office layouts and furniture arrangements.

[0008] "Specific features" are elements or attributes contained in the image data, such as color, material, furniture type, and layout.

[0009] "Tagging" refers to the process of assigning identifying labels to elements with specific characteristics.

[0010] A "generation request" refers to data that requests the generation of a new proposed image by specifying elements of the office layout desired by the user.

[0011] "New Image" refers to the visual representation of the office layout created by the AI ​​model based on the generation request.

[0012] "Evaluation feedback" refers to information in which a user expresses an evaluation opinion or a request for improvement regarding a proposed image.

[0013] "User" refers to a customer who receives an office layout proposal and a person who evaluates the proposal.

[0014] "Server" refers to the computer system that manages the entire proposed system and processes data.

[0015] "Terminal" refers to an input and display device that enables a user to use services provided by a server.

[0016] An "AI model" refers to software that uses artificial intelligence algorithms to analyze and learn from data to perform a specific task (in this case, image generation). [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[0039] Basic system configuration

[0040] The server has the ability to collect image data based on past cases and tag this data with specific features. The tagged data is stored in a database. The user uses their device to specify the elements to be used in the proposed office layout and sends the request to the server. The server generates new proposed images based on the received request and provides them to the user. The user provides evaluation feedback on the generated images, and the server updates the AI ​​model based on this feedback.

[0041] Program processing overview

[0042] Data collection and tagging (server side)

[0043] The server collects image data from past proposal photos and product catalogs. This image data is tagged with specific characteristics (e.g., color, material, furniture type, layout, etc.) using a pre-configured AI algorithm. Once tagged, the image data is stored in a database and used for subsequent image generation.

[0044] Receiving a generation request (user / device side)

[0045] The user specifies elements for a specific office layout proposal through the terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be considered. These elements are entered into a request form on the terminal and sent to the server.

[0046] Image generation (server side)

[0047] The server analyzes the received request and searches a database for relevant image data based on the specified elements. The AI ​​model generates a new suggested image based on the search results and provides it to the user. The generated image combines the specified elements to achieve a realistic visual representation.

[0048] Image confirmation and evaluation (user device side)

[0049] The generated proposed image is sent to the user's device, where the user can review it and provide feedback on the proposed image, including evaluation of specific elements (e.g., "I prefer a slightly brighter color").

[0050] Feedback and AI model updates (server side)

[0051] The server receives and analyzes feedback from users. Based on the analysis results, the parameters of the AI ​​model are updated, improving the accuracy of image generation from the next time onwards and enabling it to better adapt to user requests.

[0052] Specific examples

[0053] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for material images that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. If the user rates the design as "very appealing," the feedback information is passed on to the server and used to make future proposals.

[0054] This allows users to quickly and accurately receive proposals for their desired office layout, revolutionizing the office furniture proposal process.

[0055] The processing flow will be explained below.

[0056] Program processing flow

[0057] Data collection and tagging (server side)

[0058] Step 1:

[0059] The server collects image data from past proposal photos and product catalogs, either from existing databases within the company or from external sources.

[0060] Step 2:

[0061] The server stores the collected image data in a database, where it is properly categorized and organized by past cases for quick retrieval later.

[0062] Step 3:

[0063] The server uses AI algorithms to tag the stored image data with specific characteristics, such as metadata like "color: blue," "material: wood," or "type: desk."

[0064] Receiving a generation request (user / device side)

[0065] Step 4:

[0066] The user starts up the terminal and accesses the interface of the proposed system.

[0067] Step 5:

[0068] The user inputs the elements of the office layout they want to propose (color, material, type of furniture, layout, etc.) into a form on the interface.

[0069] Step 6:

[0070] The device sends the user's input to the server as a generation request, where the input elements are properly formatted and converted into generation request data.

[0071] Image generation (server side)

[0072] Step 7:

[0073] The server analyzes the creation request received from the terminal and extracts the specified element.

[0074] Step 8:

[0075] The server searches the image data in its database for material that matches the specified elements, and the search results are based on the tagged metadata.

[0076] Step 9:

[0077] The server uses AI models to generate new suggested images from the search results, where the retrieved elements are combined to create realistic visual representations.

[0078] Image confirmation and evaluation (user device side)

[0079] Step 10:

[0080] The server transmits the generated proposed image to the user's terminal.

[0081] Step 11:

[0082] The terminal displays the received proposed image and provides a confirmation screen to the user.

[0083] Step 12:

[0084] The user checks the proposed image and enters feedback in the "Evaluation" form. For example, the user can write specific evaluation opinions such as "I like the color of the wooden desk, but I would like the carpet to be lighter."

[0085] Step 13:

[0086] The terminal transmits the user's evaluation feedback to the server, which converts the feedback data into a format that allows the evaluation content to be easily analyzed.

[0087] Feedback and AI model updates (server side)

[0088] Step 14:

[0089] The server analyzes the evaluation feedback received from the users, and the analysis results include the user's satisfaction level and improvement requests for each element.

[0090] Step 15:

[0091] The server updates the parameters of the AI ​​model based on the analyzed feedback, allowing it to make suggestions that better match the user's requests when generating the next image.

[0092] Step 16:

[0093] The server stores the updated AI model and new learning data in a database, ready for use in the next proposal process.

[0094] Through the above steps, the office furniture suggestion process is carried out efficiently and effectively, and a system that can quickly respond to user requests is realized.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] Modern office layout proposals require efficient, low-cost generation of high-quality proposal images. However, conventional methods have the drawback of taking time to generate images and making it difficult to precisely meet user requirements. Furthermore, improving the proposed images based on user feedback requires time and effort, making it difficult to respond in real time.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for providing the generated image to the user, means for receiving feedback from the user, means for analyzing the feedback and updating parameters of the AI ​​model, and means for generating the next image using the updated AI model. This enables the generation of high-quality proposed images that quickly and accurately meet user requirements and real-time improvement of the AI ​​model based on the feedback.

[0100] "Image data" refers to image information stored in digital format, and is the basis for generating the proposed image.

[0101] "Features" refers to characteristics such as color, material, furniture type, and layout contained within the image data.

[0102] "Tagging" refers to the process of identifying characteristics of collected image data and attaching that information as metadata.

[0103] A "generation request" refers to a request to generate a new image based on user-proposed elements.

[0104] "Server" refers to the computer system that collects image data, tags it, analyzes generation requests, generates images, receives feedback, and updates the AI ​​model.

[0105] "Database" refers to a structured collection of data that allows for efficient storage and retrieval of collected image data and tagged features.

[0106] An "AI model" is a model trained using machine learning algorithms and used to generate new images and update them based on feedback.

[0107] "Feedback" refers to the evaluations and comments provided by users on the generated proposed images, and is information used to improve the AI ​​model.

[0108] "Generative AI model" refers to an AI algorithm that uses AI technology to generate new images based on user requests and a database.

[0109] A "prompt sentence" refers to a sentence for inputting a specific generation request to the system, which includes an instruction for generating a proposed image.

[0110] This invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[0111] The server collects image data based on past cases and tags specific features. The collected image data is then automatically tagged with specific features such as color, material, furniture type, and layout using an AI algorithm installed on the server. This AI algorithm uses AI frameworks such as TensorFlow and PyTorch. Once tagged, the images are stored in a database on the server.

[0112] The user inputs specific office layout elements into a request form using a terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be specified. When the user presses the submit button on the request form, the request data is sent to the server.

[0113] The server analyzes the request received from the user and searches for relevant image data from a database based on the specified elements. The server then uses the search results and a generative AI model to generate a new proposed image. This generative AI model uses AI technologies such as GAN (generative adversarial network) and VQ-VAE (vector quantization variational autoencoder). The generated proposed image is then sent from the server to the user's device.

[0114] The user can check the sent proposed image on their own device. Then, the user can input evaluation feedback for the proposed image. For example, the user can input comments such as "I like this design" or "I prefer brighter colors." When the user sends the evaluation feedback, the feedback data is sent to the server.

[0115] The server analyzes the feedback received from the user and updates the parameters of the AI ​​model based on the analysis results. This analysis can be performed using data analysis tools such as Python and Pandas. The updated AI model is reflected in the next image generation, enabling the generation of high-quality proposed images that are more suited to the user's requests.

[0116] As a concrete example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and presses the submit button, sending the request data to the server. The server searches past case data for image data containing these elements, and the generative AI model generates a new proposed image. The generated image is sent to the user's device. If the user reviews the image and rates it as "I really like the design," this feedback information is passed on to the server. Based on this feedback, the algorithm for generating proposed images from the next time onwards is improved.

[0117] An example of a prompt might be, "Please suggest an open-plan office design with wooden desks and blue carpet." Based on this prompt, the server can generate an appropriate proposal image.

[0118] As a result, the office furniture proposal process is carried out quickly and accurately, and high-quality proposals are made in response to user requests.

[0119] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0120] Step 1: Data collection and tagging (server side)

[0121] The server collects image data from past case photos and product catalogs. Specifically, it acquires images using web crawling technology and manual uploads. It then uses AI frameworks such as TensorFlow and PyTorch to automatically tag the acquired image data with specific features, such as color, material, furniture type, and layout. It uses raw image data as input, performs feature extraction based on this, and generates tagged image data as output. This data is then stored in a database for efficient search.

[0122] Step 2: Accepting user requests (user / device side)

[0123] The user inputs the desired office layout elements into the request form on the terminal. Specifically, they input elements such as "wooden desks," "blue carpet," and "open layout," and then press the submit button. The input includes a prompt text that the user enters, which is sent from the terminal as an HTTP request to the server. The server receives the request and prepares to analyze it.

[0124] Step 3: Request analysis and image data search (server side)

[0125] The server analyzes the received request and searches the database for relevant image data based on the specified element (prompt sentence). This analysis uses natural language processing technology, and uses SQL queries and Elasticsearch to extract relevant data from the database. The input is the prompt sentence and the database, and by searching based on this, the server obtains relevant image data as the output.

[0126] Step 4: Creating a new image (server side)

[0127] The server uses a generative AI model (such as GAN or VQ-VAE) to generate a new proposed image based on the image data obtained as a search result. Specifically, related image data is used as input, and the AI ​​model analyzes this data to generate a new, high-quality image that includes the desired elements. The search result image data is the input, and the generated proposed image is the output.

[0128] Step 5: Providing generated images (server / device side)

[0129] The server sends the generated proposed image to the user's device. Image data is sent to the user's device as an HTTP response, and the user can view the image. The generated proposed image is the input, and it is obtained as the output to be provided to the user via HTTP.

[0130] Step 6: Evaluation and feedback of proposed images (user / device side)

[0131] The user checks the proposed image on the terminal and inputs their evaluation feedback. Specifically, they input their evaluation such as "I like the design" or "Brighter colors would be better" and press the send button. The user's evaluation feedback is input and is sent to the server.

[0132] Step 7: Analyze feedback and update the AI ​​model (server side)

[0133] The server analyzes the received feedback and updates the parameters of the AI ​​model based on the results. Data analysis tools such as Python and Pandas are used for the analysis, and a dataset is prepared for retraining the model. User feedback is used as input, and the AI ​​model is updated based on this, obtaining an output that will be reflected in subsequent image generation.

[0134] Through these steps, the system can generate high-quality suggested images that quickly and precisely meet user requirements, and improve the AI ​​model in real time based on feedback.

[0135] (Application example 1)

[0136] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0137] The design and layout of modern office environments are becoming increasingly complex, requiring rapid, high-quality proposals that meet user requirements. However, proposing office furniture and layouts is costly and difficult to do efficiently. Viewing physical samples and actual layouts is particularly time-consuming and requires a great deal of trial and error before users are satisfied. Furthermore, conventional systems struggle to effectively utilize user feedback, resulting in the time required to improve the accuracy of the proposed images.

[0138] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0139] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received request, means for receiving feedback from the user for evaluating the generated image, means for updating the AI ​​model based on the feedback, and means for the user to visualize the generated image in real time in a virtual environment. This allows the user to visualize the office layout in a virtual environment in real time, making the proposal process efficient and low-cost. Furthermore, by effectively utilizing user feedback, the accuracy of the AI ​​model can be improved, enabling faster, higher-quality proposals.

[0140] "Image data" is digital data containing visual information, such as photographs and illustrations of office furniture and layout.

[0141] "Tagging" is the process of adding labels to collected image data to identify specific features (e.g., color, material, furniture type, layout, etc.).

[0142] A "generation request" refers to a request made by a user to specify desired office furniture and layout elements and to request the server to generate a new proposed image based on those elements.

[0143] "Feedback" is evaluation information provided by the user regarding the generated proposed image, and includes comments regarding color, layout, design, and the like, for example.

[0144] An "AI model" is a machine learning algorithm or network that uses artificial intelligence technology to analyze data and generate new proposed images.

[0145] "Real-time visualization" is a technology that allows users to instantly see the office layout and furniture placement in a virtual environment.

[0146] A "virtual environment" is a virtual space or screen generated by computer simulation that provides a realistic visual experience despite not being real.

[0147] The present invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[0148] Basic system configuration

[0149] server

[0150] The server has a means for collecting image data based on past cases. This image data includes photographs and illustrations of office furniture and its layout, and is digital data containing visual information. The collected image data is given labels (tagging means) to identify specific features (e.g., color, material, furniture type, layout, etc.).

[0151] The server has a means for receiving a generation request based on elements of an office layout specified by a user. The user inputs the elements using a smartphone, smart glasses, or a head-mounted display, and sends the request to the server.

[0152] The received generation request is analyzed, and related image data is searched for in the database based on the specified elements, and the AI ​​model generates a new proposed image based on that (image generation means). The generated image is sent to the user's device and visualized in real time in a virtual environment.

[0153] The server has a means for receiving user feedback on the generated images, including comments on color, placement, design, etc. This feedback is analyzed and used to update the parameters of the AI ​​model, improving the accuracy of future suggestions (AI model update means).

[0154] Hardware and software used

[0155] Hardware: On the client side, it uses a smartphone, smart glasses, or a head-mounted display. On the server side, it uses a high-performance server or cloud infrastructure.

[0156] Software: Deep learning frameworks such as TensorFlow and PyTorch are used for training the AI ​​model and generating images, and frameworks such as Flask and FastAPI are used for API communication.

[0157] Specific examples

[0158] As a concrete example, consider the case where a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form via a smartphone, smart glasses, or head-mounted display, and sends it to the server. The server searches past case data for material images that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can view it in real time in a virtual environment.

[0159] Next, the user provides feedback on the proposed image, such as "I really like the design." This feedback information is sent to the server, which analyzes it and updates the parameters of the AI ​​model. This improves the accuracy of image generation from the next time onwards, enabling the AI ​​model to respond more quickly and accurately to user requests.

[0160] Prompt Sentence Examples

[0161] Featured elements: wooden desk, blue carpet, open layout

[0162] Feedback: I really like the design

[0163] This system will enable users to quickly and accurately receive suggestions for their desired office layout, and is expected to revolutionize the office furniture suggestion process.

[0164] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0165] Step 1:

[0166] The user specifies the elements of the proposed office layout. Using a smartphone, smart glasses, or a head-mounted display, the user enters elements such as "wooden desks," "blue carpet," and "open layout" into a request form. This input data is then sent to the server.

[0167] Step 2:

[0168] The server analyzes the generation request received from the user. Based on the received element data (e.g., "wooden desk," "blue carpet," "open layout"), the server searches for related image data from past case studies. Here, an AI model is used to identify related materials based on the specified elements.

[0169] Step 3:

[0170] The server generates a proposed image of a new office layout based on the identified material data. It uses an AI model (e.g., a generative artificial network (GAN)) to combine related image data and generate a highly accurate image. The generated image data is then sent to the user's device.

[0171] Step 4:

[0172] The user checks the proposed image displayed on the device. The user visualizes the office layout in real time in a virtual environment using a smartphone, smart glasses, or a head-mounted display. Here, real-time visualization allows the user to see how realistic the proposed image will look.

[0173] Step 5:

[0174] The user provides evaluation feedback for the proposed image. For example, if the user evaluates the design as "I really like it," this feedback information is sent to the server via the terminal.

[0175] Step 6:

[0176] The server analyzes the feedback received from the user. The server adjusts and updates the parameters of the AI ​​model based on the feedback. Specifically, it uses the feedback data to perform re-learning and parameter optimization to improve the accuracy of the AI ​​model. This improves the accuracy of the proposed image generation from the next time onwards, making it more adaptable to user requests.

[0177] Through these steps, users can quickly and accurately receive suggestions for their desired office layout. The server and AI model use feedback to continuously improve, so the suggestion process becomes more accurate over time.

[0178] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0179] The present invention relates to a system that makes the office furniture recommendation process more effective by combining an emotion engine that recognizes the user's emotions. This system is operated in cooperation between four parties: a server, a terminal, a user, and the emotion engine.

[0180] Basic system configuration

[0181] The server has the ability to collect image data based on past cases and tag this data with specific features. The tagged data is stored in a database. The user uses their device to specify the elements to be used in the proposed office layout and sends the request to the server. The server generates new proposed images based on the received request and provides them to the user. The user provides evaluation feedback on the generated images, and the server uses this to update the AI ​​model. Furthermore, an emotion engine analyzes the user's emotions and reflects this data in the imaging process to improve the quality of the proposals.

[0182] Program processing overview

[0183] Data collection and tagging (server side)

[0184] The server collects image data from past proposal photos and product catalogs. This image data is tagged with specific characteristics (e.g., color, material, furniture type, layout, etc.) using pre-defined AI algorithms. Once tagged, the image data is stored in a database and used for subsequent image generation.

[0185] Receiving a generation request (user / device side)

[0186] The user specifies elements for a specific office layout proposal through the terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be considered. These elements are entered into a request form on the terminal and sent to the server.

[0187] Image generation (server side)

[0188] The server analyzes the received request and searches a database for relevant image data based on the specified elements. The AI ​​model generates a new suggested image based on the search results and provides it to the user. The generated image combines the specified elements to achieve a realistic visual representation.

[0189] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0190] When a user checks the proposed image, the emotion engine analyzes the user's facial expressions and voice using the device's built-in camera and microphone. For example, if the user is smiling or has a dissatisfied expression, emotional data is collected in real time.

[0191] The emotion engine analyzes this data to identify emotions such as happiness, surprise, or confusion. The server then uses this emotion data to adjust the image generation process. For example, if the user responds well to bright colors, the server will use more bright colors in the next suggestions.

[0192] Image confirmation and evaluation (user device side)

[0193] The generated proposed image is sent to the user's device, where the user can review it. The user then provides feedback on the proposed image, including evaluations of specific elements (e.g., "I prefer a slightly brighter color"). The user's emotional state is also included as part of the evaluation.

[0194] Feedback and AI model updates (server side)

[0195] The server receives and analyzes user feedback (including emotion engine data). Based on the analysis results, the parameters of the AI ​​model are updated. This improves the accuracy of image generation from the next time onwards, allowing it to better adapt to user requests.

[0196] Specific examples

[0197] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for image elements that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. The emotion engine recognizes the user's smiling expression through the device's camera and sends it to the server. The server uses this emotion data to adjust its next proposal to increase elements that evoke positive emotions.

[0198] This allows users to quickly and accurately receive proposals for their desired office layout, further revolutionizing the office furniture proposal process.

[0199] The processing flow will be explained below.

[0200] Program processing flow

[0201] Data collection and tagging (server side)

[0202] Step 1:

[0203] The server collects image data from past proposal case photos and product catalogs, typically collected automatically from existing company databases or external sources.

[0204] Step 2:

[0205] The server categorizes the collected image data and stores it in a database. Data is categorized by project name, time period, type, etc.

[0206] Step 3:

[0207] The server uses AI algorithms to tag the stored image data with specific features (e.g., color, material, furniture type, layout, etc.) The tagging is done automatically, and the model is updated regularly.

[0208] Receiving a generation request (user / device side)

[0209] Step 4:

[0210] The user starts up the terminal and accesses the interface of the proposed system.

[0211] Step 5:

[0212] Users input the elements of the office layout they want to propose (color, material, furniture type, layout, etc.) into a form on the interface. For example, "wooden desks," "blue carpet," and "open layout" are examples.

[0213] Step 6:

[0214] The terminal converts the user's input content into structured generated request data and transmits it to the server.

[0215] Image generation (server side)

[0216] Step 7:

[0217] The server analyzes the creation request data received from the terminal and identifies the specified element.

[0218] Step 8:

[0219] The server searches the image data in its database for material that matches the specified elements, extracting the relevant images based on tagged metadata.

[0220] Step 9:

[0221] The server uses the AI ​​model to generate new suggested images from the search results, compositing the images to properly reflect the elements specified by the user.

[0222] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0223] Step 10:

[0224] When the user checks the proposed image, an emotion engine is activated that uses the device's built-in camera and microphone to collect the user's facial expressions and voice.

[0225] Step 11:

[0226] The emotion engine analyzes the user's facial expression data and voice tone in real time to extract emotional data, such as smiles, furrowed brows, and changes in voice tone.

[0227] Step 12:

[0228] The emotion engine sends the analyzed emotion data to the server, which is then classified into categories such as "happiness," "surprise," and "confusion."

[0229] Step 13:

[0230] The server dynamically adjusts the elements of the proposed images based on the emotional data. For example, if the user responds positively to bright colors, the server will increase the number of bright colors in the next proposed image.

[0231] Image confirmation and evaluation (user device side)

[0232] Step 14:

[0233] The server transmits the generated proposed image to the user's terminal.

[0234] Step 15:

[0235] The terminal displays the received proposed image and provides a confirmation screen to the user.

[0236] Step 16:

[0237] The user checks the proposed image and enters feedback in the "Evaluation" form. For example, the user might write a specific evaluation opinion such as "I like the color of the wooden desk, but I would like the carpet to be lighter."

[0238] Step 17:

[0239] The terminal transmits the user's evaluation feedback to the server, which converts the feedback data into a format that makes it easy to analyze the evaluation content.

[0240] Feedback and AI model updates (server side)

[0241] Step 18:

[0242] The server analyzes the evaluation feedback received from the users, and the analysis results include the user's satisfaction level and improvement requests for each element.

[0243] Step 19:

[0244] The server updates the parameters of the AI ​​model based on the analyzed feedback data, enabling it to make suggestions that better match the user's requests when generating the next image.

[0245] Step 20:

[0246] The server stores the updated AI model and new learning data in a database, ready for use in the next proposal process.

[0247] The above steps will enable quick and accurate office layout proposals that take user emotions into consideration, further revolutionizing the office furniture proposal process.

[0248] Example 2

[0249] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0250] Existing office layout proposal systems often fail to take user emotions into account, resulting in proposals that often fail to satisfy users. Furthermore, if the generated proposed images do not match the user's preferences, the AI ​​model is not updated sufficiently based on the feedback, resulting in a lack of improvement in proposal accuracy in the future.

[0251] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0252] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for collecting emotional data from the user's facial expressions and voice using a terminal, means for analyzing the emotional data and reflecting the emotional data in the generation process, means for receiving feedback from the user to evaluate the generated image, and means for updating the AI ​​model based on the feedback. This enables office layout proposals that take the user's emotions into consideration, thereby improving the accuracy of proposals and user satisfaction.

[0253] "Image data based on past cases" refers to image files collected from past performance data and product catalogs that can be used as reference for office layout proposals.

[0254] A "specific feature" is an identifiable attribute such as color, material, furniture type, layout, etc. in the image data.

[0255] "Tagging" is the process of adding specific features as labels to image data.

[0256] A "generation request" is a request sent to the server based on the elements of the office layout proposed by the user.

[0257] A "new image" is a proposed image generated using an AI model in response to a user's generation request.

[0258] A "terminal" is an electronic device such as a computer, smartphone, or tablet that is operated by a user.

[0259] "Emotion data" refers to information about the user's emotional state analyzed from their facial expressions and voice.

[0260] An "emotion engine" is software that analyzes collected emotional data and identifies the user's emotions.

[0261] "Feedback" refers to user evaluations and opinions regarding the generated proposed image.

[0262] An "AI model" is a mathematical model that uses artificial intelligence technology to learn and make predictions based on input data.

[0263] "Parameters" are internal settings used to adjust the functions and behavior of an AI model.

[0264] "Analysis" is the process of examining collected data and extracting necessary information.

[0265] The present invention relates to a system that recognizes and reflects the user's emotions, thereby making the office furniture recommendation process more effective. This system operates in cooperation with four parties: a server, a terminal, a user, and an emotion engine.

[0266] Basic system configuration

[0267] Server: The server collects image data from past office proposals and product catalogs. This image data is tagged with specific features using an AI algorithm. The tagged data is then stored in a database.

[0268] Terminal: The user specifies elements for a specific office layout proposal through the terminal. For example, elements could be "wooden desks," "blue carpet," and "open layout." The terminal then sends these elements to the server.

[0269] Emotion Engine: When a user reviews a proposed image, the device's built-in camera and microphone are used to collect the user's facial expressions and voice. The emotion engine analyzes this data and identifies the user's emotions. This emotion data is sent to the server and reflected in the image generation process.

[0270] AI model: The server uses the AI ​​model to generate new suggested images based on the request and emotion data received from the user. The generated images are sent to the device and provided to the user. The user reviews the suggested images and provides feedback. The server updates the parameters of the AI ​​model based on this feedback, improving the accuracy of future suggestions.

[0271] Hardware and software used

[0272] Hardware: Servers, databases, user devices (PCs, tablets, smartphones), cameras, microphones

[0273] Software: Image processing libraries (e.g., OpenCV), machine learning models (e.g., TensorFlow), sentiment analysis libraries (e.g., Microsoft Azure Emotion API), cloud storage APIs, RESTful APIs

[0274] Specific examples

[0275] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for image elements that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. The emotion engine recognizes the user's smiling expression through the device's camera and sends it to the server. The server uses this emotion data to adjust its next proposal to increase elements that evoke positive emotions.

[0276] Prompt Sentence Examples

[0277] “If a user specifies wooden desks, blue carpet, and an open layout, generate proposed images of office layouts that include these elements, evaluate the user’s reactions using an emotion engine, and reflect them in your next proposal.”

[0278] According to the present invention, a user can quickly and accurately receive a proposal for a desired office layout, and the office furniture proposal process can be carried out more efficiently.

[0279] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0280] Step 1: Data collection and tagging (server side)

[0281] The server collects image data from past office proposal cases and product catalogs. Specifically, it uses a cloud storage API to download image files from multiple data sources. Next, the server uses an image processing library (e.g., OpenCV) and a machine learning model (e.g., TensorFlow) to tag the image data with specific features (e.g., color, material, furniture type, layout, etc.). Through this process, the image data is tagged with feature tags and stored in a database.

[0282] Input: Past case photos, product catalog images

[0283] Output: Image data with feature tags (stored in a database)

[0284] Step 2: Receiving the generation request (user / device side)

[0285] A user specifies elements of a particular office layout using a dedicated application on their device. For example, they input elements such as "wooden desks," "blue carpet," and "open layout" into an input form. The device generates an HTTP request based on the input elements and sends it to the server using a RESTful API.

[0286] Input: User requested elements (wooden desk, blue carpet, open layout)

[0287] Output: HTTP request sent to the server

[0288] Step 3: Image generation (server side)

[0289] The server analyzes the received request data using its analysis engine and extracts the request content. It then generates an SQL query to search the database for image data with the specified characteristics. It then uses an AI model (e.g., GANs - Generative Adversarial Networks) to generate new suggested images from the search results and sends them to the user's device as an HTTP response.

[0290] Input: Request data from the user

[0291] Output: Generated proposal image (sent to device)

[0292] Step 4: Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0293] When a user reviews a proposed image generated on their device, the device's built-in camera and microphone are used to collect the user's facial expressions and voice. The device then uses an emotion analysis library (e.g., Microsoft Azure Emotion API) to identify the user's emotions and sends that data to the server. The server then adjusts the parameters of the AI ​​model based on this emotion data and reflects it in the image generation process from the next time onwards.

[0294] Input: User's facial expression and voice data

[0295] Output: Analyzed emotion data (sent to server)

[0296] Step 5: Check and evaluate the image (user device side)

[0297] The user reviews the proposed images on their device and provides evaluation feedback. The feedback on the proposed images is entered into an input form and sent from the device to the server as an HTTP request.

[0298] Input: User rating feedback

[0299] Output: Feedback data sent to the server

[0300] Step 6: Feedback and AI model updates (server side)

[0301] The server receives user feedback and emotion data and analyzes it using an analysis engine. Based on the analysis results, it uses a machine learning algorithm (e.g., gradient descent) to readjust the parameters of the AI ​​model and improve the accuracy of subsequent image generation.

[0302] Input: User rating feedback, emotion data

[0303] Output: Updated AI model parameters

[0304] This allows the user to receive more accurate office layout proposals.

[0305] (Application example 2)

[0306] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0307] Conventional content delivery systems do not take into account the user's emotional state when making suggestions, making it difficult to appropriately provide content that the user wants to watch. Furthermore, feedback is based solely on ratings, and cannot respond to real-time emotional changes during viewing, making it impossible to provide optimal content to individual users in a timely manner. The present invention aims to solve these problems and realize more personalized content delivery based on the user's emotions.

[0308] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0309] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for receiving feedback from the user for evaluating the generated image, means for updating the AI ​​model based on the feedback, means for acquiring user emotional data, and means for adjusting the image generation process based on the acquired emotional data. This makes it possible to analyze the user's emotional state in real time and provide optimal content based on that.

[0310] "Past cases" refer to previously recorded data or information, specifically records of content that a user has previously viewed or rated.

[0311] "Image data" refers to digitized visual information, and specifically includes thumbnails and video frames of content.

[0312] "Tagging" is the process of adding specific attributes or characteristics to data, making it easier to search and classify.

[0313] "Suggestion" refers to the system's action of providing content and layout to the user, showing the optimal options based on the user's requests and conditions.

[0314] A "generation request" is a request that includes information that allows a user to request the system to generate content or an image.

[0315] "Image generation" refers to the process of creating a new image based on specified elements, and is done by combining the necessary information from AI models and databases.

[0316] "Feedback" refers to users' evaluations and reactions to the generated content and services, and this data is used to improve the system.

[0317] An "AI model" is an algorithm built using machine learning and deep learning, which is used to analyze data and make new suggestions and generate images.

[0318] "Emotion data" is data that indicates the emotional state of the user, obtained from facial expressions, voice, behavior, and the like.

[0319] "Tuning" refers to changing the parameters of a system or model in order to improve accuracy or performance.

[0320] The present invention is a system that analyzes a user's emotional data and proposes personalized content based on that data. This system is operated through cooperation between four parties: a server, a terminal, a user, and an emotion engine.

[0321] Basic system configuration

[0322] Data collection and tagging (server side)

[0323] The server collects past viewing history and rating data and tags it with specific characteristics, such as the genre of the video viewed, actors, emotional triggers, and ratings, etc. This data is then stored in a database and used for subsequent content suggestions.

[0324] Receiving a generation request (user / device side)

[0325] A user uses a device to request suggestions for specific content, such as "I want to watch funny videos." These elements are entered into a request form on the smartphone and sent to the server.

[0326] Video suggestions (server side)

[0327] The server analyzes the received request and searches a database for relevant video data based on the specified elements. It then uses a generative AI model to generate new suggested videos and provide them to the user. The generated videos combine the specified elements to provide a high-quality visual experience.

[0328] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0329] While a user is watching a video, the emotion engine analyzes the user's facial expressions and voice using the device's built-in camera and microphone. For example, if the user is smiling, emotion data is collected in real time. The emotion engine analyzes this data and identifies emotions such as joy or displeasure. The server then adjusts the next content suggestion process based on the emotion data.

[0330] Video review and evaluation (user device side)

[0331] The generated suggested videos are sent to the user's device, where the user watches them. The user then provides evaluation feedback for the suggested videos, including evaluations of specific elements and the user's emotional state.

[0332] Feedback and AI model updates (server side)

[0333] The server receives and analyzes feedback from users (including emotional data). Based on the analysis results, the parameters of the generative AI model are readjusted. This improves the accuracy of video suggestions from the next time onwards, enabling the provision of content that is more suited to the user's preferences.

[0334] Hardware and software used

[0335] Hardware

[0336] Smartphone: Watching videos, using the camera and microphone

[0337] Server: Database, generative AI model

[0338] software

[0339] OpenCV: Smartphone camera control

[0340] dlib: face detection

[0341] EmotionRecognizer: A virtual emotion analysis library

[0342] requests: HTTP request library

[0343] Specific examples

[0344] When a user requests "I want to watch a funny video," the server searches for related comedy videos based on past viewing history and rating data, and generates new suggested videos using a generative AI model. The generated video is sent to the user's smartphone, and the user begins watching it. The smartphone's camera and microphone recognize the user's smile in real time through the emotion engine. As a result, the next suggested video will contain many elements that are likely to make the user smile.

[0345] Example prompt sentence:

[0346] "I'm stressed out today, so I want to watch a relaxing video."

[0347] "Please make a video that will cheer you up."

[0348] "I've been feeling sad lately, so I want to watch a video that will make me feel cheerful."

[0349] Based on these prompts, the system can provide content that best suits the user's emotional state in real time.

[0350] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0351] Step 1:

[0352] The server collects past viewing history and rating data. The specific input is the history of videos watched by users and their rating data, and based on this, tagged images and metadata are stored in a database. The output is a tagged database.

[0353] Data processing involves adding specific characteristics such as genre, actors, emotional triggers, and ratings to the collected data.

[0354] Step 2:

[0355] The user inputs a content suggestion request using a terminal. The specific input is a desired content, such as "I want to watch a funny video," and this is sent to the server. The output is the request information.

[0356] Data processing involves analyzing the user's request and sending the necessary information to the server.

[0357] Step 3:

[0358] The server analyzes the user's request and searches for relevant video data from the database. The specific input is the request content and database information. The output is raw data for the generative AI model.

[0359] Data processing involves searching for related materials based on the specified elements and extracting data.

[0360] Step 4:

[0361] The server generates new suggested videos using a generative AI model. The specific input is the retrieved material data. The output is the newly generated suggested videos.

[0362] Data processing is the process of generating a video based on the material data searched by the AI ​​model.

[0363] Step 5:

[0364] The user watches the generated suggested videos on their device. The specific input is the generated suggested videos. The output is the user's viewing experience and feedback information.

[0365] No data processing is required, and viewing experience and feedback are provided.

[0366] Step 6:

[0367] The device acquires the user's emotional data through a camera and microphone and analyzes it with an emotion engine. The specific input is the user's facial expression and voice. The output is the analyzed emotional data.

[0368] Data processing involves using an emotion engine to analyze emotion data and identify the emotional state.

[0369] Step 7:

[0370] The terminal transmits the user's emotional data to the server. The specific input is the emotional data analyzed by the emotion engine. The output is the emotional data transmitted to the server.

[0371] No data processing is required and the data is transmitted.

[0372] Step 8:

[0373] The server analyzes the user's feedback and emotion data and updates the AI ​​model. The specific input is the user's feedback data and emotion data. The output is the updated AI model.

[0374] Data processing involves analyzing feedback data and emotional data and readjusting the parameters of the AI ​​model.

[0375] Step 9:

[0376] The server adjusts the next content suggestion process. The specific inputs are the updated AI model and the user's emotional data. The output is a new content suggestion process.

[0377] In terms of data processing, the next proposal will be made more personalized based on the newly obtained data.

[0378] Through these steps, it becomes possible to analyze user emotions in real time and provide optimal content based on that.

[0379] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0380] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0381] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0382] [Second embodiment]

[0383] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0384] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0385] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0386] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0387] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0388] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0389] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0390] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0391] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0392] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0393] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0394] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0395] The present invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[0396] Basic system configuration

[0397] The server has the ability to collect image data based on past cases and tag this data with specific features. The tagged data is stored in a database. The user uses their device to specify the elements to be used in the proposed office layout and sends the request to the server. The server generates new proposed images based on the received request and provides them to the user. The user provides evaluation feedback on the generated images, and the server updates the AI ​​model based on this feedback.

[0398] Program processing overview

[0399] Data collection and tagging (server side)

[0400] The server collects image data from past proposal photos and product catalogs. This image data is tagged with specific characteristics (e.g., color, material, furniture type, layout, etc.) using a pre-configured AI algorithm. Once tagged, the image data is stored in a database and used for subsequent image generation.

[0401] Receiving a generation request (user / device side)

[0402] The user specifies elements for a specific office layout proposal through the terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be considered. These elements are entered into a request form on the terminal and sent to the server.

[0403] Image generation (server side)

[0404] The server analyzes the received request and searches a database for relevant image data based on the specified elements. The AI ​​model generates a new suggested image based on the search results and provides it to the user. The generated image combines the specified elements to achieve a realistic visual representation.

[0405] Image confirmation and evaluation (user device side)

[0406] The generated proposed image is sent to the user's device, where the user can review it and provide feedback on the proposed image, including evaluation of specific elements (e.g., "I prefer a slightly brighter color").

[0407] Feedback and AI model updates (server side)

[0408] The server receives and analyzes feedback from users. Based on the analysis results, the parameters of the AI ​​model are updated, improving the accuracy of image generation from the next time onwards and enabling it to better adapt to user requests.

[0409] Specific examples

[0410] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for material images that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. If the user rates the design as "very appealing," the feedback information is passed on to the server and used to make future proposals.

[0411] This allows users to quickly and accurately receive proposals for their desired office layout, revolutionizing the office furniture proposal process.

[0412] The processing flow will be explained below.

[0413] Program processing flow

[0414] Data collection and tagging (server side)

[0415] Step 1:

[0416] The server collects image data from past proposal photos and product catalogs, either from existing databases within the company or from external sources.

[0417] Step 2:

[0418] The server stores the collected image data in a database, where it is properly categorized and organized by past cases for quick retrieval later.

[0419] Step 3:

[0420] The server uses AI algorithms to tag the stored image data with specific characteristics, such as metadata like "color: blue," "material: wood," or "type: desk."

[0421] Receiving a generation request (user / device side)

[0422] Step 4:

[0423] The user starts up the terminal and accesses the interface of the proposed system.

[0424] Step 5:

[0425] The user inputs the elements of the office layout they want to propose (color, material, type of furniture, layout, etc.) into a form on the interface.

[0426] Step 6:

[0427] The device sends the user's input to the server as a generation request, where the input elements are properly formatted and converted into generation request data.

[0428] Image generation (server side)

[0429] Step 7:

[0430] The server analyzes the creation request received from the terminal and extracts the specified element.

[0431] Step 8:

[0432] The server searches the image data in its database for material that matches the specified elements, and the search results are based on the tagged metadata.

[0433] Step 9:

[0434] The server uses AI models to generate new suggested images from the search results, where the retrieved elements are combined to create realistic visual representations.

[0435] Image confirmation and evaluation (user device side)

[0436] Step 10:

[0437] The server transmits the generated proposed image to the user's terminal.

[0438] Step 11:

[0439] The terminal displays the received proposed image and provides a confirmation screen to the user.

[0440] Step 12:

[0441] The user checks the proposed image and enters feedback in the "Evaluation" form. For example, the user can write specific evaluation opinions such as "I like the color of the wooden desk, but I would like the carpet to be lighter."

[0442] Step 13:

[0443] The terminal transmits the user's evaluation feedback to the server, which converts the feedback data into a format that allows the evaluation content to be easily analyzed.

[0444] Feedback and AI model updates (server side)

[0445] Step 14:

[0446] The server analyzes the evaluation feedback received from the users, and the analysis results include the user's satisfaction level and improvement requests for each element.

[0447] Step 15:

[0448] The server updates the parameters of the AI ​​model based on the analyzed feedback, allowing it to make suggestions that better match the user's requests when generating the next image.

[0449] Step 16:

[0450] The server stores the updated AI model and new learning data in a database, ready for use in the next proposal process.

[0451] Through the above steps, the office furniture suggestion process is carried out efficiently and effectively, and a system that can quickly respond to user requests is realized.

[0452] Example 1

[0453] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0454] Modern office layout proposals require efficient, low-cost generation of high-quality proposal images. However, conventional methods have the drawback of taking time to generate images and making it difficult to precisely meet user requirements. Furthermore, improving the proposed images based on user feedback requires time and effort, making it difficult to respond in real time.

[0455] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0456] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for providing the generated image to the user, means for receiving feedback from the user, means for analyzing the feedback and updating parameters of the AI ​​model, and means for generating the next image using the updated AI model. This enables the generation of high-quality proposed images that quickly and accurately meet user requirements and real-time improvement of the AI ​​model based on the feedback.

[0457] "Image data" refers to image information stored in digital format, and is the basis for generating the proposed image.

[0458] "Features" refers to characteristics such as color, material, furniture type, and layout contained within the image data.

[0459] "Tagging" refers to the process of identifying characteristics of collected image data and attaching that information as metadata.

[0460] A "generation request" refers to a request to generate a new image based on user-proposed elements.

[0461] "Server" refers to the computer system that collects image data, tags it, analyzes generation requests, generates images, receives feedback, and updates the AI ​​model.

[0462] "Database" refers to a structured collection of data that allows for efficient storage and retrieval of collected image data and tagged features.

[0463] An "AI model" is a model trained using machine learning algorithms and used to generate new images and update them based on feedback.

[0464] "Feedback" refers to the evaluations and comments provided by users on the generated proposed images, and is information used to improve the AI ​​model.

[0465] "Generative AI model" refers to an AI algorithm that uses AI technology to generate new images based on user requests and a database.

[0466] A "prompt sentence" refers to a sentence for inputting a specific generation request to the system, which includes an instruction for generating a proposed image.

[0467] This invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[0468] The server collects image data based on past cases and tags specific features. The collected image data is then automatically tagged with specific features such as color, material, furniture type, and layout using an AI algorithm installed on the server. This AI algorithm uses AI frameworks such as TensorFlow and PyTorch. Once tagged, the images are stored in a database on the server.

[0469] The user inputs specific office layout elements into a request form using a terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be specified. When the user presses the submit button on the request form, the request data is sent to the server.

[0470] The server analyzes the request received from the user and searches for relevant image data from a database based on the specified elements. The server then uses the search results and a generative AI model to generate a new proposed image. This generative AI model uses AI technologies such as GAN (generative adversarial network) and VQ-VAE (vector quantization variational autoencoder). The generated proposed image is then sent from the server to the user's device.

[0471] The user can check the sent proposed image on their own device. Then, the user can input evaluation feedback for the proposed image. For example, the user can input comments such as "I like this design" or "I prefer brighter colors." When the user sends the evaluation feedback, the feedback data is sent to the server.

[0472] The server analyzes the feedback received from the user and updates the parameters of the AI ​​model based on the analysis results. This analysis can be performed using data analysis tools such as Python and Pandas. The updated AI model is reflected in the next image generation, enabling the generation of high-quality proposed images that are more suited to the user's requests.

[0473] As a concrete example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and presses the submit button, sending the request data to the server. The server searches past case data for image data containing these elements, and the generative AI model generates a new proposed image. The generated image is sent to the user's device. If the user reviews the image and rates it as "I really like the design," this feedback information is passed on to the server. Based on this feedback, the algorithm for generating proposed images from the next time onwards is improved.

[0474] An example of a prompt might be, "Please suggest an open-plan office design with wooden desks and blue carpet." Based on this prompt, the server can generate an appropriate proposal image.

[0475] As a result, the office furniture proposal process is carried out quickly and accurately, and high-quality proposals are made in response to user requests.

[0476] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0477] Step 1: Data collection and tagging (server side)

[0478] The server collects image data from past case photos and product catalogs. Specifically, it acquires images using web crawling technology and manual uploads. It then uses AI frameworks such as TensorFlow and PyTorch to automatically tag the acquired image data with specific features, such as color, material, furniture type, and layout. It uses raw image data as input, performs feature extraction based on this, and generates tagged image data as output. This data is then stored in a database for efficient search.

[0479] Step 2: Accepting user requests (user / device side)

[0480] The user inputs the desired office layout elements into the request form on the terminal. Specifically, they input elements such as "wooden desks," "blue carpet," and "open layout," and then press the submit button. The input includes a prompt text that the user enters, which is sent from the terminal as an HTTP request to the server. The server receives the request and prepares to analyze it.

[0481] Step 3: Request analysis and image data search (server side)

[0482] The server analyzes the received request and searches the database for relevant image data based on the specified element (prompt sentence). This analysis uses natural language processing technology, and uses SQL queries and Elasticsearch to extract relevant data from the database. The input is the prompt sentence and the database, and by searching based on this, the server obtains relevant image data as the output.

[0483] Step 4: Creating a new image (server side)

[0484] The server uses a generative AI model (such as GAN or VQ-VAE) to generate a new proposed image based on the image data obtained as a search result. Specifically, related image data is used as input, and the AI ​​model analyzes this data to generate a new, high-quality image that includes the desired elements. The search result image data is the input, and the generated proposed image is the output.

[0485] Step 5: Providing generated images (server / device side)

[0486] The server sends the generated proposed image to the user's device. Image data is sent to the user's device as an HTTP response, and the user can view the image. The generated proposed image is the input, and it is obtained as the output to be provided to the user via HTTP.

[0487] Step 6: Evaluation and feedback of proposed images (user / device side)

[0488] The user checks the proposed image on the terminal and inputs their evaluation feedback. Specifically, they input their evaluation such as "I like the design" or "Brighter colors would be better" and press the send button. The user's evaluation feedback is input and is sent to the server.

[0489] Step 7: Analyze feedback and update the AI ​​model (server side)

[0490] The server analyzes the received feedback and updates the parameters of the AI ​​model based on the results. Data analysis tools such as Python and Pandas are used for the analysis, and a dataset is prepared for retraining the model. User feedback is used as input, and the AI ​​model is updated based on this, obtaining an output that will be reflected in subsequent image generation.

[0491] Through these steps, the system can generate high-quality suggested images that quickly and precisely meet user requirements, and improve the AI ​​model in real time based on feedback.

[0492] (Application example 1)

[0493] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0494] The design and layout of modern office environments are becoming increasingly complex, requiring rapid, high-quality proposals that meet user requirements. However, proposing office furniture and layouts is costly and difficult to do efficiently. Viewing physical samples and actual layouts is particularly time-consuming and requires a great deal of trial and error before users are satisfied. Furthermore, conventional systems struggle to effectively utilize user feedback, resulting in the time required to improve the accuracy of the proposed images.

[0495] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0496] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received request, means for receiving feedback from the user for evaluating the generated image, means for updating the AI ​​model based on the feedback, and means for the user to visualize the generated image in real time in a virtual environment. This allows the user to visualize the office layout in a virtual environment in real time, making the proposal process efficient and low-cost. Furthermore, by effectively utilizing user feedback, the accuracy of the AI ​​model can be improved, enabling faster, higher-quality proposals.

[0497] "Image data" is digital data containing visual information, such as photographs and illustrations of office furniture and layout.

[0498] "Tagging" is the process of adding labels to collected image data to identify specific features (e.g., color, material, furniture type, layout, etc.).

[0499] A "generation request" refers to a request made by a user to specify desired office furniture and layout elements and to request the server to generate a new proposed image based on those elements.

[0500] "Feedback" is evaluation information provided by the user regarding the generated proposed image, and includes comments regarding color, layout, design, and the like, for example.

[0501] An "AI model" is a machine learning algorithm or network that uses artificial intelligence technology to analyze data and generate new proposed images.

[0502] "Real-time visualization" is a technology that allows users to instantly see the office layout and furniture placement in a virtual environment.

[0503] A "virtual environment" is a virtual space or screen generated by computer simulation that provides a realistic visual experience despite not being real.

[0504] The present invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[0505] Basic system configuration

[0506] server

[0507] The server has a means for collecting image data based on past cases. This image data includes photographs and illustrations of office furniture and its layout, and is digital data containing visual information. The collected image data is given labels (tagging means) to identify specific features (e.g., color, material, furniture type, layout, etc.).

[0508] The server has a means for receiving a generation request based on elements of an office layout specified by a user. The user inputs the elements using a smartphone, smart glasses, or a head-mounted display, and sends the request to the server.

[0509] The received generation request is analyzed, and related image data is searched for in the database based on the specified elements, and the AI ​​model generates a new proposed image based on that (image generation means). The generated image is sent to the user's device and visualized in real time in a virtual environment.

[0510] The server has a means for receiving user feedback on the generated images, including comments on color, placement, design, etc. This feedback is analyzed and used to update the parameters of the AI ​​model, improving the accuracy of future suggestions (AI model update means).

[0511] Hardware and software used

[0512] Hardware: On the client side, it uses a smartphone, smart glasses, or a head-mounted display. On the server side, it uses a high-performance server or cloud infrastructure.

[0513] Software: Deep learning frameworks such as TensorFlow and PyTorch are used for training the AI ​​model and generating images, and frameworks such as Flask and FastAPI are used for API communication.

[0514] Specific examples

[0515] As a concrete example, consider the case where a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form via a smartphone, smart glasses, or head-mounted display, and sends it to the server. The server searches past case data for material images that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can view it in real time in a virtual environment.

[0516] Next, the user provides feedback on the proposed image, such as "I really like the design." This feedback information is sent to the server, which analyzes it and updates the parameters of the AI ​​model. This improves the accuracy of image generation from the next time onwards, enabling the AI ​​model to respond more quickly and accurately to user requests.

[0517] Prompt Sentence Examples

[0518] Featured elements: wooden desk, blue carpet, open layout

[0519] Feedback: I really like the design

[0520] This system will enable users to quickly and accurately receive suggestions for their desired office layout, and is expected to revolutionize the office furniture suggestion process.

[0521] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0522] Step 1:

[0523] The user specifies the elements of the proposed office layout. Using a smartphone, smart glasses, or a head-mounted display, the user enters elements such as "wooden desks," "blue carpet," and "open layout" into a request form. This input data is then sent to the server.

[0524] Step 2:

[0525] The server analyzes the generation request received from the user. Based on the received element data (e.g., "wooden desk," "blue carpet," "open layout"), the server searches for related image data from past case studies. Here, an AI model is used to identify related materials based on the specified elements.

[0526] Step 3:

[0527] The server generates a proposed image of a new office layout based on the identified material data. It uses an AI model (e.g., a generative artificial network (GAN)) to combine related image data and generate a highly accurate image. The generated image data is then sent to the user's device.

[0528] Step 4:

[0529] The user checks the proposed image displayed on the device. The user visualizes the office layout in real time in a virtual environment using a smartphone, smart glasses, or a head-mounted display. Here, real-time visualization allows the user to see how realistic the proposed image will look.

[0530] Step 5:

[0531] The user provides evaluation feedback for the proposed image. For example, if the user evaluates the design as "I really like it," this feedback information is sent to the server via the terminal.

[0532] Step 6:

[0533] The server analyzes the feedback received from the user. The server adjusts and updates the parameters of the AI ​​model based on the feedback. Specifically, it uses the feedback data to perform re-learning and parameter optimization to improve the accuracy of the AI ​​model. This improves the accuracy of the proposed image generation from the next time onwards, making it more adaptable to user requests.

[0534] Through these steps, users can quickly and accurately receive suggestions for their desired office layout. The server and AI model use feedback to continuously improve, so the suggestion process becomes more accurate over time.

[0535] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0536] The present invention relates to a system that makes the office furniture recommendation process more effective by combining an emotion engine that recognizes the user's emotions. This system is operated in cooperation between four parties: a server, a terminal, a user, and the emotion engine.

[0537] Basic system configuration

[0538] The server has the ability to collect image data based on past cases and tag this data with specific features. The tagged data is stored in a database. The user uses their device to specify the elements to be used in the proposed office layout and sends the request to the server. The server generates new proposed images based on the received request and provides them to the user. The user provides evaluation feedback on the generated images, and the server uses this to update the AI ​​model. Furthermore, an emotion engine analyzes the user's emotions and reflects this data in the imaging process to improve the quality of the proposals.

[0539] Program processing overview

[0540] Data collection and tagging (server side)

[0541] The server collects image data from past proposal photos and product catalogs. This image data is tagged with specific characteristics (e.g., color, material, furniture type, layout, etc.) using pre-defined AI algorithms. Once tagged, the image data is stored in a database and used for subsequent image generation.

[0542] Receiving a generation request (user / device side)

[0543] The user specifies elements for a specific office layout proposal through the terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be considered. These elements are entered into a request form on the terminal and sent to the server.

[0544] Image generation (server side)

[0545] The server analyzes the received request and searches a database for relevant image data based on the specified elements. The AI ​​model generates a new suggested image based on the search results and provides it to the user. The generated image combines the specified elements to achieve a realistic visual representation.

[0546] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0547] When a user checks the proposed image, the emotion engine analyzes the user's facial expressions and voice using the device's built-in camera and microphone. For example, if the user is smiling or has a dissatisfied expression, emotional data is collected in real time.

[0548] The emotion engine analyzes this data to identify emotions such as happiness, surprise, or confusion. The server then uses this emotion data to adjust the image generation process. For example, if the user responds well to bright colors, the server will use more bright colors in the next suggestions.

[0549] Image confirmation and evaluation (user device side)

[0550] The generated proposed image is sent to the user's device, where the user can review it. The user then provides feedback on the proposed image, including evaluations of specific elements (e.g., "I prefer a slightly brighter color"). The user's emotional state is also included as part of the evaluation.

[0551] Feedback and AI model updates (server side)

[0552] The server receives and analyzes user feedback (including emotion engine data). Based on the analysis results, the parameters of the AI ​​model are updated. This improves the accuracy of image generation from the next time onwards, allowing it to better adapt to user requests.

[0553] Specific examples

[0554] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for image elements that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. The emotion engine recognizes the user's smiling expression through the device's camera and sends it to the server. The server uses this emotion data to adjust its next proposal to increase elements that evoke positive emotions.

[0555] This allows users to quickly and accurately receive proposals for their desired office layout, further revolutionizing the office furniture proposal process.

[0556] The processing flow will be explained below.

[0557] Program processing flow

[0558] Data collection and tagging (server side)

[0559] Step 1:

[0560] The server collects image data from past proposal case photos and product catalogs, typically collected automatically from existing company databases or external sources.

[0561] Step 2:

[0562] The server categorizes the collected image data and stores it in a database. Data is categorized by project name, time period, type, etc.

[0563] Step 3:

[0564] The server uses AI algorithms to tag the stored image data with specific features (e.g., color, material, furniture type, layout, etc.) The tagging is done automatically, and the model is updated regularly.

[0565] Receiving a generation request (user / device side)

[0566] Step 4:

[0567] The user starts up the terminal and accesses the interface of the proposed system.

[0568] Step 5:

[0569] Users input the elements of the office layout they want to propose (color, material, furniture type, layout, etc.) into a form on the interface. For example, "wooden desks," "blue carpet," and "open layout" are examples.

[0570] Step 6:

[0571] The terminal converts the user's input content into structured generated request data and transmits it to the server.

[0572] Image generation (server side)

[0573] Step 7:

[0574] The server analyzes the creation request data received from the terminal and identifies the specified element.

[0575] Step 8:

[0576] The server searches the image data in its database for material that matches the specified elements, extracting the relevant images based on tagged metadata.

[0577] Step 9:

[0578] The server uses the AI ​​model to generate new suggested images from the search results, compositing the images to properly reflect the elements specified by the user.

[0579] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0580] Step 10:

[0581] When the user checks the proposed image, an emotion engine is activated that uses the device's built-in camera and microphone to collect the user's facial expressions and voice.

[0582] Step 11:

[0583] The emotion engine analyzes the user's facial expression data and voice tone in real time to extract emotional data, such as smiles, furrowed brows, and changes in voice tone.

[0584] Step 12:

[0585] The emotion engine sends the analyzed emotion data to the server, which is then classified into categories such as "happiness," "surprise," and "confusion."

[0586] Step 13:

[0587] The server dynamically adjusts the elements of the proposed images based on the emotional data. For example, if the user responds positively to bright colors, the server will increase the number of bright colors in the next proposed image.

[0588] Image confirmation and evaluation (user device side)

[0589] Step 14:

[0590] The server transmits the generated proposed image to the user's terminal.

[0591] Step 15:

[0592] The terminal displays the received proposed image and provides a confirmation screen to the user.

[0593] Step 16:

[0594] The user checks the proposed image and enters feedback in the "Evaluation" form. For example, the user might write a specific evaluation opinion such as "I like the color of the wooden desk, but I would like the carpet to be lighter."

[0595] Step 17:

[0596] The terminal transmits the user's evaluation feedback to the server, which converts the feedback data into a format that makes it easy to analyze the evaluation content.

[0597] Feedback and AI model updates (server side)

[0598] Step 18:

[0599] The server analyzes the evaluation feedback received from the users, and the analysis results include the user's satisfaction level and improvement requests for each element.

[0600] Step 19:

[0601] The server updates the parameters of the AI ​​model based on the analyzed feedback data, enabling it to make suggestions that better match the user's requests when generating the next image.

[0602] Step 20:

[0603] The server stores the updated AI model and new learning data in a database, ready for use in the next proposal process.

[0604] The above steps will enable quick and accurate office layout proposals that take user emotions into consideration, further revolutionizing the office furniture proposal process.

[0605] Example 2

[0606] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0607] Existing office layout proposal systems often fail to take user emotions into account, resulting in proposals that often fail to satisfy users. Furthermore, if the generated proposed images do not match the user's preferences, the AI ​​model is not updated sufficiently based on the feedback, resulting in a lack of improvement in proposal accuracy in the future.

[0608] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0609] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for collecting emotional data from the user's facial expressions and voice using a terminal, means for analyzing the emotional data and reflecting the emotional data in the generation process, means for receiving feedback from the user to evaluate the generated image, and means for updating the AI ​​model based on the feedback. This enables office layout proposals that take the user's emotions into consideration, thereby improving the accuracy of proposals and user satisfaction.

[0610] "Image data based on past cases" refers to image files collected from past performance data and product catalogs that can be used as reference for office layout proposals.

[0611] A "specific feature" is an identifiable attribute such as color, material, furniture type, layout, etc. in the image data.

[0612] "Tagging" is the process of adding specific features as labels to image data.

[0613] A "generation request" is a request sent to the server based on the elements of the office layout proposed by the user.

[0614] A "new image" is a proposed image generated using an AI model in response to a user's generation request.

[0615] A "terminal" is an electronic device such as a computer, smartphone, or tablet that is operated by a user.

[0616] "Emotion data" refers to information about the user's emotional state analyzed from their facial expressions and voice.

[0617] An "emotion engine" is software that analyzes collected emotional data and identifies the user's emotions.

[0618] "Feedback" refers to user evaluations and opinions regarding the generated proposed image.

[0619] An "AI model" is a mathematical model that uses artificial intelligence technology to learn and make predictions based on input data.

[0620] "Parameters" are internal settings used to adjust the functions and behavior of an AI model.

[0621] "Analysis" is the process of examining collected data and extracting necessary information.

[0622] The present invention relates to a system that recognizes and reflects the user's emotions, thereby making the office furniture recommendation process more effective. This system operates in cooperation with four parties: a server, a terminal, a user, and an emotion engine.

[0623] Basic system configuration

[0624] Server: The server collects image data from past office proposals and product catalogs. This image data is tagged with specific features using an AI algorithm. The tagged data is then stored in a database.

[0625] Terminal: The user specifies elements for a specific office layout proposal through the terminal. For example, elements could be "wooden desks," "blue carpet," and "open layout." The terminal then sends these elements to the server.

[0626] Emotion Engine: When a user reviews a proposed image, the device's built-in camera and microphone are used to collect the user's facial expressions and voice. The emotion engine analyzes this data and identifies the user's emotions. This emotion data is sent to the server and reflected in the image generation process.

[0627] AI model: The server uses the AI ​​model to generate new suggested images based on the request and emotion data received from the user. The generated images are sent to the device and provided to the user. The user reviews the suggested images and provides feedback. The server updates the parameters of the AI ​​model based on this feedback, improving the accuracy of future suggestions.

[0628] Hardware and software used

[0629] Hardware: Servers, databases, user devices (PCs, tablets, smartphones), cameras, microphones

[0630] Software: Image processing libraries (e.g., OpenCV), machine learning models (e.g., TensorFlow), sentiment analysis libraries (e.g., Microsoft Azure Emotion API), cloud storage APIs, RESTful APIs

[0631] Specific examples

[0632] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for image elements that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. The emotion engine recognizes the user's smiling expression through the device's camera and sends it to the server. The server uses this emotion data to adjust its next proposal to increase elements that evoke positive emotions.

[0633] Prompt Sentence Examples

[0634] “If a user specifies wooden desks, blue carpet, and an open layout, generate proposed images of office layouts that include these elements, evaluate the user’s reactions using an emotion engine, and reflect them in your next proposal.”

[0635] According to the present invention, a user can quickly and accurately receive a proposal for a desired office layout, and the office furniture proposal process can be carried out more efficiently.

[0636] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0637] Step 1: Data collection and tagging (server side)

[0638] The server collects image data from past office proposal cases and product catalogs. Specifically, it uses a cloud storage API to download image files from multiple data sources. Next, the server uses an image processing library (e.g., OpenCV) and a machine learning model (e.g., TensorFlow) to tag the image data with specific features (e.g., color, material, furniture type, layout, etc.). Through this process, the image data is tagged with feature tags and stored in a database.

[0639] Input: Past case photos, product catalog images

[0640] Output: Image data with feature tags (stored in a database)

[0641] Step 2: Receiving the generation request (user / device side)

[0642] A user specifies elements of a particular office layout using a dedicated application on their device. For example, they input elements such as "wooden desks," "blue carpet," and "open layout" into an input form. The device generates an HTTP request based on the input elements and sends it to the server using a RESTful API.

[0643] Input: User requested elements (wooden desk, blue carpet, open layout)

[0644] Output: HTTP request sent to the server

[0645] Step 3: Image generation (server side)

[0646] The server analyzes the received request data using its analysis engine and extracts the request content. It then generates an SQL query to search the database for image data with the specified characteristics. It then uses an AI model (e.g., GANs - Generative Adversarial Networks) to generate new suggested images from the search results and sends them to the user's device as an HTTP response.

[0647] Input: Request data from the user

[0648] Output: Generated proposal image (sent to device)

[0649] Step 4: Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0650] When a user reviews a proposed image generated on their device, the device's built-in camera and microphone are used to collect the user's facial expressions and voice. The device then uses an emotion analysis library (e.g., Microsoft Azure Emotion API) to identify the user's emotions and sends that data to the server. The server then adjusts the parameters of the AI ​​model based on this emotion data and reflects it in the image generation process from the next time onwards.

[0651] Input: User's facial expression and voice data

[0652] Output: Analyzed emotion data (sent to server)

[0653] Step 5: Check and evaluate the image (user device side)

[0654] The user reviews the proposed images on their device and provides evaluation feedback. The feedback on the proposed images is entered into an input form and sent from the device to the server as an HTTP request.

[0655] Input: User rating feedback

[0656] Output: Feedback data sent to the server

[0657] Step 6: Feedback and AI model updates (server side)

[0658] The server receives user feedback and emotion data and analyzes it using an analysis engine. Based on the analysis results, it uses a machine learning algorithm (e.g., gradient descent) to readjust the parameters of the AI ​​model and improve the accuracy of subsequent image generation.

[0659] Input: User rating feedback, emotion data

[0660] Output: Updated AI model parameters

[0661] This allows the user to receive more accurate office layout proposals.

[0662] (Application example 2)

[0663] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0664] Conventional content delivery systems do not take into account the user's emotional state when making suggestions, making it difficult to appropriately provide content that the user wants to watch. Furthermore, feedback is based solely on ratings, and cannot respond to real-time emotional changes during viewing, making it impossible to provide optimal content to individual users in a timely manner. The present invention aims to solve these problems and realize more personalized content delivery based on the user's emotions.

[0665] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0666] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for receiving feedback from the user for evaluating the generated image, means for updating the AI ​​model based on the feedback, means for acquiring user emotional data, and means for adjusting the image generation process based on the acquired emotional data. This makes it possible to analyze the user's emotional state in real time and provide optimal content based on that.

[0667] "Past cases" refer to previously recorded data or information, specifically records of content that a user has previously viewed or rated.

[0668] "Image data" refers to digitized visual information, and specifically includes thumbnails and video frames of content.

[0669] "Tagging" is the process of adding specific attributes or characteristics to data, making it easier to search and classify.

[0670] "Suggestion" refers to the system's action of providing content and layout to the user, showing the optimal options based on the user's requests and conditions.

[0671] A "generation request" is a request that includes information that allows a user to request the system to generate content or an image.

[0672] "Image generation" refers to the process of creating a new image based on specified elements, and is done by combining the necessary information from AI models and databases.

[0673] "Feedback" refers to users' evaluations and reactions to the generated content and services, and this data is used to improve the system.

[0674] An "AI model" is an algorithm built using machine learning and deep learning, which is used to analyze data and make new suggestions and generate images.

[0675] "Emotion data" is data that indicates the emotional state of the user, obtained from facial expressions, voice, behavior, and the like.

[0676] "Tuning" refers to changing the parameters of a system or model in order to improve accuracy or performance.

[0677] The present invention is a system that analyzes a user's emotional data and proposes personalized content based on that data. This system is operated through cooperation between four parties: a server, a terminal, a user, and an emotion engine.

[0678] Basic system configuration

[0679] Data collection and tagging (server side)

[0680] The server collects past viewing history and rating data and tags it with specific characteristics, such as the genre of the video viewed, actors, emotional triggers, and ratings, etc. This data is then stored in a database and used for subsequent content suggestions.

[0681] Receiving a generation request (user / device side)

[0682] A user uses a device to request suggestions for specific content, such as "I want to watch funny videos." These elements are entered into a request form on the smartphone and sent to the server.

[0683] Video suggestions (server side)

[0684] The server analyzes the received request and searches a database for relevant video data based on the specified elements. It then uses a generative AI model to generate new suggested videos and provide them to the user. The generated videos combine the specified elements to provide a high-quality visual experience.

[0685] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0686] While a user is watching a video, the emotion engine analyzes the user's facial expressions and voice using the device's built-in camera and microphone. For example, if the user is smiling, emotion data is collected in real time. The emotion engine analyzes this data and identifies emotions such as joy or displeasure. The server then adjusts the next content suggestion process based on the emotion data.

[0687] Video review and evaluation (user device side)

[0688] The generated suggested videos are sent to the user's device, where the user watches them. The user then provides evaluation feedback for the suggested videos, including evaluations of specific elements and the user's emotional state.

[0689] Feedback and AI model updates (server side)

[0690] The server receives and analyzes feedback from users (including emotional data). Based on the analysis results, the parameters of the generative AI model are readjusted. This improves the accuracy of video suggestions from the next time onwards, enabling the provision of content that is more suited to the user's preferences.

[0691] Hardware and software used

[0692] Hardware

[0693] Smartphone: Watching videos, using the camera and microphone

[0694] Server: Database, generative AI model

[0695] software

[0696] OpenCV: Smartphone camera control

[0697] dlib: face detection

[0698] EmotionRecognizer: A virtual emotion analysis library

[0699] requests: HTTP request library

[0700] Specific examples

[0701] When a user requests "I want to watch a funny video," the server searches for related comedy videos based on past viewing history and rating data, and generates new suggested videos using a generative AI model. The generated video is sent to the user's smartphone, and the user begins watching it. The smartphone's camera and microphone recognize the user's smile in real time through the emotion engine. As a result, the next suggested video will contain many elements that are likely to make the user smile.

[0702] Example prompt sentence:

[0703] "I'm stressed out today, so I want to watch a relaxing video."

[0704] "Please make a video that will cheer you up."

[0705] "I've been feeling sad lately, so I want to watch a video that will make me feel cheerful."

[0706] Based on these prompts, the system can provide content that best suits the user's emotional state in real time.

[0707] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0708] Step 1:

[0709] The server collects past viewing history and rating data. The specific input is the history of videos watched by users and their rating data, and based on this, tagged images and metadata are stored in a database. The output is a tagged database.

[0710] Data processing involves adding specific characteristics such as genre, actors, emotional triggers, and ratings to the collected data.

[0711] Step 2:

[0712] The user inputs a content suggestion request using a terminal. The specific input is a desired content, such as "I want to watch a funny video," and this is sent to the server. The output is the request information.

[0713] Data processing involves analyzing the user's request and sending the necessary information to the server.

[0714] Step 3:

[0715] The server analyzes the user's request and searches for relevant video data from the database. The specific input is the request content and database information. The output is raw data for the generative AI model.

[0716] Data processing involves searching for related materials based on the specified elements and extracting data.

[0717] Step 4:

[0718] The server generates new suggested videos using a generative AI model. The specific input is the retrieved material data. The output is the newly generated suggested videos.

[0719] Data processing is the process of generating a video based on the material data searched by the AI ​​model.

[0720] Step 5:

[0721] The user watches the generated suggested videos on their device. The specific input is the generated suggested videos. The output is the user's viewing experience and feedback information.

[0722] No data processing is required, and viewing experience and feedback are provided.

[0723] Step 6:

[0724] The device acquires the user's emotional data through a camera and microphone and analyzes it with an emotion engine. The specific input is the user's facial expression and voice. The output is the analyzed emotional data.

[0725] Data processing involves using an emotion engine to analyze emotion data and identify the emotional state.

[0726] Step 7:

[0727] The terminal transmits the user's emotional data to the server. The specific input is the emotional data analyzed by the emotion engine. The output is the emotional data transmitted to the server.

[0728] No data processing is required and the data is transmitted.

[0729] Step 8:

[0730] The server analyzes the user's feedback and emotion data and updates the AI ​​model. The specific input is the user's feedback data and emotion data. The output is the updated AI model.

[0731] Data processing involves analyzing feedback data and emotional data and readjusting the parameters of the AI ​​model.

[0732] Step 9:

[0733] The server adjusts the next content suggestion process. The specific inputs are the updated AI model and the user's emotional data. The output is a new content suggestion process.

[0734] In terms of data processing, the next proposal will be made more personalized based on the newly obtained data.

[0735] Through these steps, it becomes possible to analyze user emotions in real time and provide optimal content based on that.

[0736] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0737] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0738] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0739] [Third embodiment]

[0740] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0741] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0742] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0743] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0744] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0745] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0746] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0747] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0748] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0749] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0750] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0751] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0752] The present invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[0753] Basic system configuration

[0754] The server has the ability to collect image data based on past cases and tag this data with specific features. The tagged data is stored in a database. The user uses their device to specify the elements to be used in the proposed office layout and sends the request to the server. The server generates new proposed images based on the received request and provides them to the user. The user provides evaluation feedback on the generated images, and the server updates the AI ​​model based on this feedback.

[0755] Program processing overview

[0756] Data collection and tagging (server side)

[0757] The server collects image data from past proposal photos and product catalogs. This image data is tagged with specific characteristics (e.g., color, material, furniture type, layout, etc.) using a pre-configured AI algorithm. Once tagged, the image data is stored in a database and used for subsequent image generation.

[0758] Receiving a generation request (user / device side)

[0759] The user specifies elements for a specific office layout proposal through the terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be considered. These elements are entered into a request form on the terminal and sent to the server.

[0760] Image generation (server side)

[0761] The server analyzes the received request and searches a database for relevant image data based on the specified elements. The AI ​​model generates a new suggested image based on the search results and provides it to the user. The generated image combines the specified elements to achieve a realistic visual representation.

[0762] Image confirmation and evaluation (user device side)

[0763] The generated proposed image is sent to the user's device, where the user can review it and provide feedback on the proposed image, including evaluation of specific elements (e.g., "I prefer a slightly brighter color").

[0764] Feedback and AI model updates (server side)

[0765] The server receives and analyzes feedback from users. Based on the analysis results, the parameters of the AI ​​model are updated, improving the accuracy of image generation from the next time onwards and enabling it to better adapt to user requests.

[0766] Specific examples

[0767] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for material images that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. If the user rates the design as "very appealing," the feedback information is passed on to the server and used to make future proposals.

[0768] This allows users to quickly and accurately receive proposals for their desired office layout, revolutionizing the office furniture proposal process.

[0769] The processing flow will be explained below.

[0770] Program processing flow

[0771] Data collection and tagging (server side)

[0772] Step 1:

[0773] The server collects image data from past proposal photos and product catalogs, either from existing databases within the company or from external sources.

[0774] Step 2:

[0775] The server stores the collected image data in a database, where it is properly categorized and organized by past cases for quick retrieval later.

[0776] Step 3:

[0777] The server uses AI algorithms to tag the stored image data with specific characteristics, such as metadata like "color: blue," "material: wood," or "type: desk."

[0778] Receiving a generation request (user / device side)

[0779] Step 4:

[0780] The user starts up the terminal and accesses the interface of the proposed system.

[0781] Step 5:

[0782] The user inputs the elements of the office layout they want to propose (color, material, type of furniture, layout, etc.) into a form on the interface.

[0783] Step 6:

[0784] The device sends the user's input to the server as a generation request, where the input elements are properly formatted and converted into generation request data.

[0785] Image generation (server side)

[0786] Step 7:

[0787] The server analyzes the creation request received from the terminal and extracts the specified element.

[0788] Step 8:

[0789] The server searches the image data in its database for material that matches the specified elements, and the search results are based on the tagged metadata.

[0790] Step 9:

[0791] The server uses AI models to generate new suggested images from the search results, where the retrieved elements are combined to create realistic visual representations.

[0792] Image confirmation and evaluation (user device side)

[0793] Step 10:

[0794] The server transmits the generated proposed image to the user's terminal.

[0795] Step 11:

[0796] The terminal displays the received proposed image and provides a confirmation screen to the user.

[0797] Step 12:

[0798] The user checks the proposed image and enters feedback in the "Evaluation" form. For example, the user can write specific evaluation opinions such as "I like the color of the wooden desk, but I would like the carpet to be lighter."

[0799] Step 13:

[0800] The terminal transmits the user's evaluation feedback to the server, which converts the feedback data into a format that allows the evaluation content to be easily analyzed.

[0801] Feedback and AI model updates (server side)

[0802] Step 14:

[0803] The server analyzes the evaluation feedback received from the users, and the analysis results include the user's satisfaction level and improvement requests for each element.

[0804] Step 15:

[0805] The server updates the parameters of the AI ​​model based on the analyzed feedback, allowing it to make suggestions that better match the user's requests when generating the next image.

[0806] Step 16:

[0807] The server stores the updated AI model and new learning data in a database, ready for use in the next proposal process.

[0808] Through the above steps, the office furniture suggestion process is carried out efficiently and effectively, and a system that can quickly respond to user requests is realized.

[0809] Example 1

[0810] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0811] Modern office layout proposals require efficient, low-cost generation of high-quality proposal images. However, conventional methods have the drawback of taking time to generate images and making it difficult to precisely meet user requirements. Furthermore, improving the proposed images based on user feedback requires time and effort, making it difficult to respond in real time.

[0812] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0813] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for providing the generated image to the user, means for receiving feedback from the user, means for analyzing the feedback and updating parameters of the AI ​​model, and means for generating the next image using the updated AI model. This enables the generation of high-quality proposed images that quickly and accurately meet user requirements and real-time improvement of the AI ​​model based on the feedback.

[0814] "Image data" refers to image information stored in digital format, and is the basis for generating the proposed image.

[0815] "Features" refers to characteristics such as color, material, furniture type, and layout contained within the image data.

[0816] "Tagging" refers to the process of identifying characteristics of collected image data and attaching that information as metadata.

[0817] A "generation request" refers to a request to generate a new image based on user-proposed elements.

[0818] "Server" refers to the computer system that collects image data, tags it, analyzes generation requests, generates images, receives feedback, and updates the AI ​​model.

[0819] "Database" refers to a structured collection of data that allows for efficient storage and retrieval of collected image data and tagged features.

[0820] An "AI model" is a model trained using machine learning algorithms and used to generate new images and update them based on feedback.

[0821] "Feedback" refers to the evaluations and comments provided by users on the generated proposed images, and is information used to improve the AI ​​model.

[0822] "Generative AI model" refers to an AI algorithm that uses AI technology to generate new images based on user requests and a database.

[0823] A "prompt sentence" refers to a sentence for inputting a specific generation request to the system, which includes an instruction for generating a proposed image.

[0824] This invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[0825] The server collects image data based on past cases and tags specific features. The collected image data is then automatically tagged with specific features such as color, material, furniture type, and layout using an AI algorithm installed on the server. This AI algorithm uses AI frameworks such as TensorFlow and PyTorch. Once tagged, the images are stored in a database on the server.

[0826] The user inputs specific office layout elements into a request form using a terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be specified. When the user presses the submit button on the request form, the request data is sent to the server.

[0827] The server analyzes the request received from the user and searches for relevant image data from a database based on the specified elements. The server then uses the search results and a generative AI model to generate a new proposed image. This generative AI model uses AI technologies such as GAN (generative adversarial network) and VQ-VAE (vector quantization variational autoencoder). The generated proposed image is then sent from the server to the user's device.

[0828] The user can check the sent proposed image on their own device. Then, the user can input evaluation feedback for the proposed image. For example, the user can input comments such as "I like this design" or "I prefer brighter colors." When the user sends the evaluation feedback, the feedback data is sent to the server.

[0829] The server analyzes the feedback received from the user and updates the parameters of the AI ​​model based on the analysis results. This analysis can be performed using data analysis tools such as Python and Pandas. The updated AI model is reflected in the next image generation, enabling the generation of high-quality proposed images that are more suited to the user's requests.

[0830] As a concrete example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and presses the submit button, sending the request data to the server. The server searches past case data for image data containing these elements, and the generative AI model generates a new proposed image. The generated image is sent to the user's device. If the user reviews the image and rates it as "I really like the design," this feedback information is passed on to the server. Based on this feedback, the algorithm for generating proposed images from the next time onwards is improved.

[0831] An example of a prompt might be, "Please suggest an open-plan office design with wooden desks and blue carpet." Based on this prompt, the server can generate an appropriate proposal image.

[0832] As a result, the office furniture proposal process is carried out quickly and accurately, and high-quality proposals are made in response to user requests.

[0833] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0834] Step 1: Data collection and tagging (server side)

[0835] The server collects image data from past case photos and product catalogs. Specifically, it acquires images using web crawling technology and manual uploads. It then uses AI frameworks such as TensorFlow and PyTorch to automatically tag the acquired image data with specific features, such as color, material, furniture type, and layout. It uses raw image data as input, performs feature extraction based on this, and generates tagged image data as output. This data is then stored in a database for efficient search.

[0836] Step 2: Accepting user requests (user / device side)

[0837] The user inputs the desired office layout elements into the request form on the terminal. Specifically, they input elements such as "wooden desks," "blue carpet," and "open layout," and then press the submit button. The input includes a prompt text that the user enters, which is sent from the terminal as an HTTP request to the server. The server receives the request and prepares to analyze it.

[0838] Step 3: Request analysis and image data search (server side)

[0839] The server analyzes the received request and searches the database for relevant image data based on the specified element (prompt sentence). This analysis uses natural language processing technology, and uses SQL queries and Elasticsearch to extract relevant data from the database. The input is the prompt sentence and the database, and by searching based on this, the server obtains relevant image data as the output.

[0840] Step 4: Creating a new image (server side)

[0841] The server uses a generative AI model (such as GAN or VQ-VAE) to generate a new proposed image based on the image data obtained as a search result. Specifically, related image data is used as input, and the AI ​​model analyzes this data to generate a new, high-quality image that includes the desired elements. The search result image data is the input, and the generated proposed image is the output.

[0842] Step 5: Providing generated images (server / device side)

[0843] The server sends the generated proposed image to the user's device. Image data is sent to the user's device as an HTTP response, and the user can view the image. The generated proposed image is the input, and it is obtained as the output to be provided to the user via HTTP.

[0844] Step 6: Evaluation and feedback of proposed images (user / device side)

[0845] The user checks the proposed image on the terminal and inputs their evaluation feedback. Specifically, they input their evaluation such as "I like the design" or "Brighter colors would be better" and press the send button. The user's evaluation feedback is input and is sent to the server.

[0846] Step 7: Analyze feedback and update the AI ​​model (server side)

[0847] The server analyzes the received feedback and updates the parameters of the AI ​​model based on the results. Data analysis tools such as Python and Pandas are used for the analysis, and a dataset is prepared for retraining the model. User feedback is used as input, and the AI ​​model is updated based on this, obtaining an output that will be reflected in subsequent image generation.

[0848] Through these steps, the system can generate high-quality suggested images that quickly and precisely meet user requirements, and improve the AI ​​model in real time based on feedback.

[0849] (Application example 1)

[0850] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0851] The design and layout of modern office environments are becoming increasingly complex, requiring rapid, high-quality proposals that meet user requirements. However, proposing office furniture and layouts is costly and difficult to do efficiently. Viewing physical samples and actual layouts is particularly time-consuming and requires a great deal of trial and error before users are satisfied. Furthermore, conventional systems struggle to effectively utilize user feedback, resulting in the time required to improve the accuracy of the proposed images.

[0852] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0853] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received request, means for receiving feedback from the user for evaluating the generated image, means for updating the AI ​​model based on the feedback, and means for the user to visualize the generated image in real time in a virtual environment. This allows the user to visualize the office layout in a virtual environment in real time, making the proposal process efficient and low-cost. Furthermore, by effectively utilizing user feedback, the accuracy of the AI ​​model can be improved, enabling faster, higher-quality proposals.

[0854] "Image data" is digital data containing visual information, such as photographs and illustrations of office furniture and layout.

[0855] "Tagging" is the process of adding labels to collected image data to identify specific features (e.g., color, material, furniture type, layout, etc.).

[0856] A "generation request" refers to a request made by a user to specify desired office furniture and layout elements and to request the server to generate a new proposed image based on those elements.

[0857] "Feedback" is evaluation information provided by the user regarding the generated proposed image, and includes comments regarding color, layout, design, and the like, for example.

[0858] An "AI model" is a machine learning algorithm or network that uses artificial intelligence technology to analyze data and generate new proposed images.

[0859] "Real-time visualization" is a technology that allows users to instantly see the office layout and furniture placement in a virtual environment.

[0860] A "virtual environment" is a virtual space or screen generated by computer simulation that provides a realistic visual experience despite not being real.

[0861] The present invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[0862] Basic system configuration

[0863] server

[0864] The server has a means for collecting image data based on past cases. This image data includes photographs and illustrations of office furniture and its layout, and is digital data containing visual information. The collected image data is given labels (tagging means) to identify specific features (e.g., color, material, furniture type, layout, etc.).

[0865] The server has a means for receiving a generation request based on elements of an office layout specified by a user. The user inputs the elements using a smartphone, smart glasses, or a head-mounted display, and sends the request to the server.

[0866] The received generation request is analyzed, and related image data is searched for in the database based on the specified elements, and the AI ​​model generates a new proposed image based on that (image generation means). The generated image is sent to the user's device and visualized in real time in a virtual environment.

[0867] The server has a means for receiving user feedback on the generated images, including comments on color, placement, design, etc. This feedback is analyzed and used to update the parameters of the AI ​​model, improving the accuracy of future suggestions (AI model update means).

[0868] Hardware and software used

[0869] Hardware: On the client side, it uses a smartphone, smart glasses, or a head-mounted display. On the server side, it uses a high-performance server or cloud infrastructure.

[0870] Software: Deep learning frameworks such as TensorFlow and PyTorch are used for training the AI ​​model and generating images, and frameworks such as Flask and FastAPI are used for API communication.

[0871] Specific examples

[0872] As a concrete example, consider the case where a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form via a smartphone, smart glasses, or head-mounted display, and sends it to the server. The server searches past case data for material images that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can view it in real time in a virtual environment.

[0873] Next, the user provides feedback on the proposed image, such as "I really like the design." This feedback information is sent to the server, which analyzes it and updates the parameters of the AI ​​model. This improves the accuracy of image generation from the next time onwards, enabling the AI ​​model to respond more quickly and accurately to user requests.

[0874] Prompt Sentence Examples

[0875] Featured elements: wooden desk, blue carpet, open layout

[0876] Feedback: I really like the design

[0877] This system will enable users to quickly and accurately receive suggestions for their desired office layout, and is expected to revolutionize the office furniture suggestion process.

[0878] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0879] Step 1:

[0880] The user specifies the elements of the proposed office layout. Using a smartphone, smart glasses, or a head-mounted display, the user enters elements such as "wooden desks," "blue carpet," and "open layout" into a request form. This input data is then sent to the server.

[0881] Step 2:

[0882] The server analyzes the generation request received from the user. Based on the received element data (e.g., "wooden desk," "blue carpet," "open layout"), the server searches for related image data from past case studies. Here, an AI model is used to identify related materials based on the specified elements.

[0883] Step 3:

[0884] The server generates a proposed image of a new office layout based on the identified material data. It uses an AI model (e.g., a generative artificial network (GAN)) to combine related image data and generate a highly accurate image. The generated image data is then sent to the user's device.

[0885] Step 4:

[0886] The user checks the proposed image displayed on the device. The user visualizes the office layout in real time in a virtual environment using a smartphone, smart glasses, or a head-mounted display. Here, real-time visualization allows the user to see how realistic the proposed image will look.

[0887] Step 5:

[0888] The user provides evaluation feedback for the proposed image. For example, if the user evaluates the design as "I really like it," this feedback information is sent to the server via the terminal.

[0889] Step 6:

[0890] The server analyzes the feedback received from the user. The server adjusts and updates the parameters of the AI ​​model based on the feedback. Specifically, it uses the feedback data to perform re-learning and parameter optimization to improve the accuracy of the AI ​​model. This improves the accuracy of the proposed image generation from the next time onwards, making it more adaptable to user requests.

[0891] Through these steps, users can quickly and accurately receive suggestions for their desired office layout. The server and AI model use feedback to continuously improve, so the suggestion process becomes more accurate over time.

[0892] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0893] The present invention relates to a system that makes the office furniture recommendation process more effective by combining an emotion engine that recognizes the user's emotions. This system is operated in cooperation between four parties: a server, a terminal, a user, and the emotion engine.

[0894] Basic system configuration

[0895] The server has the ability to collect image data based on past cases and tag this data with specific features. The tagged data is stored in a database. The user uses their device to specify the elements to be used in the proposed office layout and sends the request to the server. The server generates new proposed images based on the received request and provides them to the user. The user provides evaluation feedback on the generated images, and the server uses this to update the AI ​​model. Furthermore, an emotion engine analyzes the user's emotions and reflects this data in the imaging process to improve the quality of the proposals.

[0896] Program processing overview

[0897] Data collection and tagging (server side)

[0898] The server collects image data from past proposal photos and product catalogs. This image data is tagged with specific characteristics (e.g., color, material, furniture type, layout, etc.) using pre-defined AI algorithms. Once tagged, the image data is stored in a database and used for subsequent image generation.

[0899] Receiving a generation request (user / device side)

[0900] The user specifies elements for a specific office layout proposal through the terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be considered. These elements are entered into a request form on the terminal and sent to the server.

[0901] Image generation (server side)

[0902] The server analyzes the received request and searches a database for relevant image data based on the specified elements. The AI ​​model generates a new suggested image based on the search results and provides it to the user. The generated image combines the specified elements to achieve a realistic visual representation.

[0903] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0904] When a user checks the proposed image, the emotion engine analyzes the user's facial expressions and voice using the device's built-in camera and microphone. For example, if the user is smiling or has a dissatisfied expression, emotional data is collected in real time.

[0905] The emotion engine analyzes this data to identify emotions such as happiness, surprise, or confusion. The server then uses this emotion data to adjust the image generation process. For example, if the user responds well to bright colors, the server will use more bright colors in the next suggestions.

[0906] Image confirmation and evaluation (user device side)

[0907] The generated proposed image is sent to the user's device, where the user can review it. The user then provides feedback on the proposed image, including evaluations of specific elements (e.g., "I prefer a slightly brighter color"). The user's emotional state is also included as part of the evaluation.

[0908] Feedback and AI model updates (server side)

[0909] The server receives and analyzes user feedback (including emotion engine data). Based on the analysis results, the parameters of the AI ​​model are updated. This improves the accuracy of image generation from the next time onwards, allowing it to better adapt to user requests.

[0910] Specific examples

[0911] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for image elements that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. The emotion engine recognizes the user's smiling expression through the device's camera and sends it to the server. The server uses this emotion data to adjust its next proposal to increase elements that evoke positive emotions.

[0912] This allows users to quickly and accurately receive proposals for their desired office layout, further revolutionizing the office furniture proposal process.

[0913] The processing flow will be explained below.

[0914] Program processing flow

[0915] Data collection and tagging (server side)

[0916] Step 1:

[0917] The server collects image data from past proposal case photos and product catalogs, typically collected automatically from existing company databases or external sources.

[0918] Step 2:

[0919] The server categorizes the collected image data and stores it in a database. Data is categorized by project name, time period, type, etc.

[0920] Step 3:

[0921] The server uses AI algorithms to tag the stored image data with specific features (e.g., color, material, furniture type, layout, etc.) The tagging is done automatically, and the model is updated regularly.

[0922] Receiving a generation request (user / device side)

[0923] Step 4:

[0924] The user starts up the terminal and accesses the interface of the proposed system.

[0925] Step 5:

[0926] Users input the elements of the office layout they want to propose (color, material, furniture type, layout, etc.) into a form on the interface. For example, "wooden desks," "blue carpet," and "open layout" are examples.

[0927] Step 6:

[0928] The terminal converts the user's input content into structured generated request data and transmits it to the server.

[0929] Image generation (server side)

[0930] Step 7:

[0931] The server analyzes the creation request data received from the terminal and identifies the specified element.

[0932] Step 8:

[0933] The server searches the image data in its database for material that matches the specified elements, extracting the relevant images based on tagged metadata.

[0934] Step 9:

[0935] The server uses the AI ​​model to generate new suggested images from the search results, compositing the images to properly reflect the elements specified by the user.

[0936] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[0937] Step 10:

[0938] When the user checks the proposed image, an emotion engine is activated that uses the device's built-in camera and microphone to collect the user's facial expressions and voice.

[0939] Step 11:

[0940] The emotion engine analyzes the user's facial expression data and voice tone in real time to extract emotional data, such as smiles, furrowed brows, and changes in voice tone.

[0941] Step 12:

[0942] The emotion engine sends the analyzed emotion data to the server, which is then classified into categories such as "happiness," "surprise," and "confusion."

[0943] Step 13:

[0944] The server dynamically adjusts the elements of the proposed images based on the emotional data. For example, if the user responds positively to bright colors, the server will increase the number of bright colors in the next proposed image.

[0945] Image confirmation and evaluation (user device side)

[0946] Step 14:

[0947] The server transmits the generated proposed image to the user's terminal.

[0948] Step 15:

[0949] The terminal displays the received proposed image and provides a confirmation screen to the user.

[0950] Step 16:

[0951] The user checks the proposed image and enters feedback in the "Evaluation" form. For example, the user might write a specific evaluation opinion such as "I like the color of the wooden desk, but I would like the carpet to be lighter."

[0952] Step 17:

[0953] The terminal transmits the user's evaluation feedback to the server, which converts the feedback data into a format that makes it easy to analyze the evaluation content.

[0954] Feedback and AI model updates (server side)

[0955] Step 18:

[0956] The server analyzes the evaluation feedback received from the users, and the analysis results include the user's satisfaction level and improvement requests for each element.

[0957] Step 19:

[0958] The server updates the parameters of the AI ​​model based on the analyzed feedback data, enabling it to make suggestions that better match the user's requests when generating the next image.

[0959] Step 20:

[0960] The server stores the updated AI model and new learning data in a database, ready for use in the next proposal process.

[0961] The above steps will enable quick and accurate office layout proposals that take user emotions into consideration, further revolutionizing the office furniture proposal process.

[0962] Example 2

[0963] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0964] Existing office layout proposal systems often fail to take user emotions into account, resulting in proposals that often fail to satisfy users. Furthermore, if the generated proposed images do not match the user's preferences, the AI ​​model is not updated sufficiently based on the feedback, resulting in a lack of improvement in proposal accuracy in the future.

[0965] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0966] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for collecting emotional data from the user's facial expressions and voice using a terminal, means for analyzing the emotional data and reflecting the emotional data in the generation process, means for receiving feedback from the user to evaluate the generated image, and means for updating the AI ​​model based on the feedback. This enables office layout proposals that take the user's emotions into consideration, thereby improving the accuracy of proposals and user satisfaction.

[0967] "Image data based on past cases" refers to image files collected from past performance data and product catalogs that can be used as reference for office layout proposals.

[0968] A "specific feature" is an identifiable attribute such as color, material, furniture type, layout, etc. in the image data.

[0969] "Tagging" is the process of adding specific features as labels to image data.

[0970] A "generation request" is a request sent to the server based on the elements of the office layout proposed by the user.

[0971] A "new image" is a proposed image generated using an AI model in response to a user's generation request.

[0972] A "terminal" is an electronic device such as a computer, smartphone, or tablet that is operated by a user.

[0973] "Emotion data" refers to information about the user's emotional state analyzed from their facial expressions and voice.

[0974] An "emotion engine" is software that analyzes collected emotional data and identifies the user's emotions.

[0975] "Feedback" refers to user evaluations and opinions regarding the generated proposed image.

[0976] An "AI model" is a mathematical model that uses artificial intelligence technology to learn and make predictions based on input data.

[0977] "Parameters" are internal settings used to adjust the functions and behavior of an AI model.

[0978] "Analysis" is the process of examining collected data and extracting necessary information.

[0979] The present invention relates to a system that recognizes and reflects the user's emotions, thereby making the office furniture recommendation process more effective. This system operates in cooperation with four parties: a server, a terminal, a user, and an emotion engine.

[0980] Basic system configuration

[0981] Server: The server collects image data from past office proposals and product catalogs. This image data is tagged with specific features using an AI algorithm. The tagged data is then stored in a database.

[0982] Terminal: The user specifies elements for a specific office layout proposal through the terminal. For example, elements could be "wooden desks," "blue carpet," and "open layout." The terminal then sends these elements to the server.

[0983] Emotion Engine: When a user reviews a proposed image, the device's built-in camera and microphone are used to collect the user's facial expressions and voice. The emotion engine analyzes this data and identifies the user's emotions. This emotion data is sent to the server and reflected in the image generation process.

[0984] AI model: The server uses the AI ​​model to generate new suggested images based on the request and emotion data received from the user. The generated images are sent to the device and provided to the user. The user reviews the suggested images and provides feedback. The server updates the parameters of the AI ​​model based on this feedback, improving the accuracy of future suggestions.

[0985] Hardware and software used

[0986] Hardware: Servers, databases, user devices (PCs, tablets, smartphones), cameras, microphones

[0987] Software: Image processing libraries (e.g., OpenCV), machine learning models (e.g., TensorFlow), sentiment analysis libraries (e.g., Microsoft Azure Emotion API), cloud storage APIs, RESTful APIs

[0988] Specific examples

[0989] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for image elements that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. The emotion engine recognizes the user's smiling expression through the device's camera and sends it to the server. The server uses this emotion data to adjust its next proposal to increase elements that evoke positive emotions.

[0990] Prompt Sentence Examples

[0991] “If a user specifies wooden desks, blue carpet, and an open layout, generate proposed images of office layouts that include these elements, evaluate the user’s reactions using an emotion engine, and reflect them in your next proposal.”

[0992] According to the present invention, a user can quickly and accurately receive a proposal for a desired office layout, and the office furniture proposal process can be carried out more efficiently.

[0993] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0994] Step 1: Data collection and tagging (server side)

[0995] The server collects image data from past office proposal cases and product catalogs. Specifically, it uses a cloud storage API to download image files from multiple data sources. Next, the server uses an image processing library (e.g., OpenCV) and a machine learning model (e.g., TensorFlow) to tag the image data with specific features (e.g., color, material, furniture type, layout, etc.). Through this process, the image data is tagged with feature tags and stored in a database.

[0996] Input: Past case photos, product catalog images

[0997] Output: Image data with feature tags (stored in a database)

[0998] Step 2: Receiving the generation request (user / device side)

[0999] A user specifies elements of a particular office layout using a dedicated application on their device. For example, they input elements such as "wooden desks," "blue carpet," and "open layout" into an input form. The device generates an HTTP request based on the input elements and sends it to the server using a RESTful API.

[1000] Input: User requested elements (wooden desk, blue carpet, open layout)

[1001] Output: HTTP request sent to the server

[1002] Step 3: Image generation (server side)

[1003] The server analyzes the received request data using its analysis engine and extracts the request content. It then generates an SQL query to search the database for image data with the specified characteristics. It then uses an AI model (e.g., GANs - Generative Adversarial Networks) to generate new suggested images from the search results and sends them to the user's device as an HTTP response.

[1004] Input: Request data from the user

[1005] Output: Generated proposal image (sent to device)

[1006] Step 4: Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[1007] When a user reviews a proposed image generated on their device, the device's built-in camera and microphone are used to collect the user's facial expressions and voice. The device then uses an emotion analysis library (e.g., Microsoft Azure Emotion API) to identify the user's emotions and sends that data to the server. The server then adjusts the parameters of the AI ​​model based on this emotion data and reflects it in the image generation process from the next time onwards.

[1008] Input: User's facial expression and voice data

[1009] Output: Analyzed emotion data (sent to server)

[1010] Step 5: Check and evaluate the image (user device side)

[1011] The user reviews the proposed images on their device and provides evaluation feedback. The feedback on the proposed images is entered into an input form and sent from the device to the server as an HTTP request.

[1012] Input: User rating feedback

[1013] Output: Feedback data sent to the server

[1014] Step 6: Feedback and AI model updates (server side)

[1015] The server receives user feedback and emotion data and analyzes it using an analysis engine. Based on the analysis results, it uses a machine learning algorithm (e.g., gradient descent) to readjust the parameters of the AI ​​model and improve the accuracy of subsequent image generation.

[1016] Input: User rating feedback, emotion data

[1017] Output: Updated AI model parameters

[1018] This allows the user to receive more accurate office layout proposals.

[1019] (Application example 2)

[1020] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1021] Conventional content delivery systems do not take into account the user's emotional state when making suggestions, making it difficult to appropriately provide content that the user wants to watch. Furthermore, feedback is based solely on ratings, and cannot respond to real-time emotional changes during viewing, making it impossible to provide optimal content to individual users in a timely manner. The present invention aims to solve these problems and realize more personalized content delivery based on the user's emotions.

[1022] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1023] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for receiving feedback from the user for evaluating the generated image, means for updating the AI ​​model based on the feedback, means for acquiring user emotional data, and means for adjusting the image generation process based on the acquired emotional data. This makes it possible to analyze the user's emotional state in real time and provide optimal content based on that.

[1024] "Past cases" refer to previously recorded data or information, specifically records of content that a user has previously viewed or rated.

[1025] "Image data" refers to digitized visual information, and specifically includes thumbnails and video frames of content.

[1026] "Tagging" is the process of adding specific attributes or characteristics to data, making it easier to search and classify.

[1027] "Suggestion" refers to the system's action of providing content and layout to the user, showing the optimal options based on the user's requests and conditions.

[1028] A "generation request" is a request that includes information that allows a user to request the system to generate content or an image.

[1029] "Image generation" refers to the process of creating a new image based on specified elements, and is done by combining the necessary information from AI models and databases.

[1030] "Feedback" refers to users' evaluations and reactions to the generated content and services, and this data is used to improve the system.

[1031] An "AI model" is an algorithm built using machine learning and deep learning, which is used to analyze data and make new suggestions and generate images.

[1032] "Emotion data" is data that indicates the emotional state of the user, obtained from facial expressions, voice, behavior, and the like.

[1033] "Tuning" refers to changing the parameters of a system or model in order to improve accuracy or performance.

[1034] The present invention is a system that analyzes a user's emotional data and proposes personalized content based on that data. This system is operated through cooperation between four parties: a server, a terminal, a user, and an emotion engine.

[1035] Basic system configuration

[1036] Data collection and tagging (server side)

[1037] The server collects past viewing history and rating data and tags it with specific characteristics, such as the genre of the video viewed, actors, emotional triggers, and ratings, etc. This data is then stored in a database and used for subsequent content suggestions.

[1038] Receiving a generation request (user / device side)

[1039] A user uses a device to request suggestions for specific content, such as "I want to watch funny videos." These elements are entered into a request form on the smartphone and sent to the server.

[1040] Video suggestions (server side)

[1041] The server analyzes the received request and searches a database for relevant video data based on the specified elements. It then uses a generative AI model to generate new suggested videos and provide them to the user. The generated videos combine the specified elements to provide a high-quality visual experience.

[1042] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[1043] While a user is watching a video, the emotion engine analyzes the user's facial expressions and voice using the device's built-in camera and microphone. For example, if the user is smiling, emotion data is collected in real time. The emotion engine analyzes this data and identifies emotions such as joy or displeasure. The server then adjusts the next content suggestion process based on the emotion data.

[1044] Video review and evaluation (user device side)

[1045] The generated suggested videos are sent to the user's device, where the user watches them. The user then provides evaluation feedback for the suggested videos, including evaluations of specific elements and the user's emotional state.

[1046] Feedback and AI model updates (server side)

[1047] The server receives and analyzes feedback from users (including emotional data). Based on the analysis results, the parameters of the generative AI model are readjusted. This improves the accuracy of video suggestions from the next time onwards, enabling the provision of content that is more suited to the user's preferences.

[1048] Hardware and software used

[1049] Hardware

[1050] Smartphone: Watching videos, using the camera and microphone

[1051] Server: Database, generative AI model

[1052] software

[1053] OpenCV: Smartphone camera control

[1054] dlib: face detection

[1055] EmotionRecognizer: A virtual emotion analysis library

[1056] requests: HTTP request library

[1057] Specific examples

[1058] When a user requests "I want to watch a funny video," the server searches for related comedy videos based on past viewing history and rating data, and generates new suggested videos using a generative AI model. The generated video is sent to the user's smartphone, and the user begins watching it. The smartphone's camera and microphone recognize the user's smile in real time through the emotion engine. As a result, the next suggested video will contain many elements that are likely to make the user smile.

[1059] Example prompt sentence:

[1060] "I'm stressed out today, so I want to watch a relaxing video."

[1061] "Please make a video that will cheer you up."

[1062] "I've been feeling sad lately, so I want to watch a video that will make me feel cheerful."

[1063] Based on these prompts, the system can provide content that best suits the user's emotional state in real time.

[1064] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1065] Step 1:

[1066] The server collects past viewing history and rating data. The specific input is the history of videos watched by users and their rating data, and based on this, tagged images and metadata are stored in a database. The output is a tagged database.

[1067] Data processing involves adding specific characteristics such as genre, actors, emotional triggers, and ratings to the collected data.

[1068] Step 2:

[1069] The user inputs a content suggestion request using a terminal. The specific input is a desired content, such as "I want to watch a funny video," and this is sent to the server. The output is the request information.

[1070] Data processing involves analyzing the user's request and sending the necessary information to the server.

[1071] Step 3:

[1072] The server analyzes the user's request and searches for relevant video data from the database. The specific input is the request content and database information. The output is raw data for the generative AI model.

[1073] Data processing involves searching for related materials based on the specified elements and extracting data.

[1074] Step 4:

[1075] The server generates new suggested videos using a generative AI model. The specific input is the retrieved material data. The output is the newly generated suggested videos.

[1076] Data processing is the process of generating a video based on the material data searched by the AI ​​model.

[1077] Step 5:

[1078] The user watches the generated suggested videos on their device. The specific input is the generated suggested videos. The output is the user's viewing experience and feedback information.

[1079] No data processing is required, and viewing experience and feedback are provided.

[1080] Step 6:

[1081] The device acquires the user's emotional data through a camera and microphone and analyzes it with an emotion engine. The specific input is the user's facial expression and voice. The output is the analyzed emotional data.

[1082] Data processing involves using an emotion engine to analyze emotion data and identify the emotional state.

[1083] Step 7:

[1084] The terminal transmits the user's emotional data to the server. The specific input is the emotional data analyzed by the emotion engine. The output is the emotional data transmitted to the server.

[1085] No data processing is required and the data is transmitted.

[1086] Step 8:

[1087] The server analyzes the user's feedback and emotion data and updates the AI ​​model. The specific input is the user's feedback data and emotion data. The output is the updated AI model.

[1088] Data processing involves analyzing feedback data and emotional data and readjusting the parameters of the AI ​​model.

[1089] Step 9:

[1090] The server adjusts the next content suggestion process. The specific inputs are the updated AI model and the user's emotional data. The output is a new content suggestion process.

[1091] In terms of data processing, the next proposal will be made more personalized based on the newly obtained data.

[1092] Through these steps, it becomes possible to analyze user emotions in real time and provide optimal content based on that.

[1093] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1094] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1095] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1096] [Fourth embodiment]

[1097] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1098] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1099] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1100] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1101] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1102] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1103] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1104] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1105] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1106] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1107] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1108] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1109] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1110] The present invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[1111] Basic system configuration

[1112] The server has the ability to collect image data based on past cases and tag this data with specific features. The tagged data is stored in a database. The user uses their device to specify the elements to be used in the proposed office layout and sends the request to the server. The server generates new proposed images based on the received request and provides them to the user. The user provides evaluation feedback on the generated images, and the server updates the AI ​​model based on this feedback.

[1113] Program processing overview

[1114] Data collection and tagging (server side)

[1115] The server collects image data from past proposal photos and product catalogs. This image data is tagged with specific characteristics (e.g., color, material, furniture type, layout, etc.) using a pre-configured AI algorithm. Once tagged, the image data is stored in a database and used for subsequent image generation.

[1116] Receiving a generation request (user / device side)

[1117] The user specifies elements for a specific office layout proposal through the terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be considered. These elements are entered into a request form on the terminal and sent to the server.

[1118] Image generation (server side)

[1119] The server analyzes the received request and searches a database for relevant image data based on the specified elements. The AI ​​model generates a new suggested image based on the search results and provides it to the user. The generated image combines the specified elements to achieve a realistic visual representation.

[1120] Image confirmation and evaluation (user device side)

[1121] The generated proposed image is sent to the user's device, where the user can review it and provide feedback on the proposed image, including evaluation of specific elements (e.g., "I prefer a slightly brighter color").

[1122] Feedback and AI model updates (server side)

[1123] The server receives and analyzes feedback from users. Based on the analysis results, the parameters of the AI ​​model are updated, improving the accuracy of image generation from the next time onwards and enabling it to better adapt to user requests.

[1124] Specific examples

[1125] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for material images that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. If the user rates the design as "very appealing," the feedback information is passed on to the server and used to make future proposals.

[1126] This allows users to quickly and accurately receive proposals for their desired office layout, revolutionizing the office furniture proposal process.

[1127] The processing flow will be explained below.

[1128] Program processing flow

[1129] Data collection and tagging (server side)

[1130] Step 1:

[1131] The server collects image data from past proposal photos and product catalogs, either from existing databases within the company or from external sources.

[1132] Step 2:

[1133] The server stores the collected image data in a database, where it is properly categorized and organized by past cases for quick retrieval later.

[1134] Step 3:

[1135] The server uses AI algorithms to tag the stored image data with specific characteristics, such as metadata like "color: blue," "material: wood," or "type: desk."

[1136] Receiving a generation request (user / device side)

[1137] Step 4:

[1138] The user starts up the terminal and accesses the interface of the proposed system.

[1139] Step 5:

[1140] The user inputs the elements of the office layout they want to propose (color, material, type of furniture, layout, etc.) into a form on the interface.

[1141] Step 6:

[1142] The device sends the user's input to the server as a generation request, where the input elements are properly formatted and converted into generation request data.

[1143] Image generation (server side)

[1144] Step 7:

[1145] The server analyzes the creation request received from the terminal and extracts the specified element.

[1146] Step 8:

[1147] The server searches the image data in its database for material that matches the specified elements, and the search results are based on the tagged metadata.

[1148] Step 9:

[1149] The server uses AI models to generate new suggested images from the search results, where the retrieved elements are combined to create realistic visual representations.

[1150] Image confirmation and evaluation (user device side)

[1151] Step 10:

[1152] The server transmits the generated proposed image to the user's terminal.

[1153] Step 11:

[1154] The terminal displays the received proposed image and provides a confirmation screen to the user.

[1155] Step 12:

[1156] The user checks the proposed image and enters feedback in the "Evaluation" form. For example, the user can write specific evaluation opinions such as "I like the color of the wooden desk, but I would like the carpet to be lighter."

[1157] Step 13:

[1158] The terminal transmits the user's evaluation feedback to the server, which converts the feedback data into a format that allows the evaluation content to be easily analyzed.

[1159] Feedback and AI model updates (server side)

[1160] Step 14:

[1161] The server analyzes the evaluation feedback received from the users, and the analysis results include the user's satisfaction level and improvement requests for each element.

[1162] Step 15:

[1163] The server updates the parameters of the AI ​​model based on the analyzed feedback, allowing it to make suggestions that better match the user's requests when generating the next image.

[1164] Step 16:

[1165] The server stores the updated AI model and new learning data in a database, ready for use in the next proposal process.

[1166] Through the above steps, the office furniture suggestion process is carried out efficiently and effectively, and a system that can quickly respond to user requests is realized.

[1167] Example 1

[1168] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1169] Modern office layout proposals require efficient, low-cost generation of high-quality proposal images. However, conventional methods have the drawback of taking time to generate images and making it difficult to precisely meet user requirements. Furthermore, improving the proposed images based on user feedback requires time and effort, making it difficult to respond in real time.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1171] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for providing the generated image to the user, means for receiving feedback from the user, means for analyzing the feedback and updating parameters of the AI ​​model, and means for generating the next image using the updated AI model. This enables the generation of high-quality proposed images that quickly and accurately meet user requirements and real-time improvement of the AI ​​model based on the feedback.

[1172] "Image data" refers to image information stored in digital format, and is the basis for generating the proposed image.

[1173] "Features" refers to characteristics such as color, material, furniture type, and layout contained within the image data.

[1174] "Tagging" refers to the process of identifying characteristics of collected image data and attaching that information as metadata.

[1175] A "generation request" refers to a request to generate a new image based on user-proposed elements.

[1176] "Server" refers to the computer system that collects image data, tags it, analyzes generation requests, generates images, receives feedback, and updates the AI ​​model.

[1177] "Database" refers to a structured collection of data that allows for efficient storage and retrieval of collected image data and tagged features.

[1178] An "AI model" is a model trained using machine learning algorithms and used to generate new images and update them based on feedback.

[1179] "Feedback" refers to the evaluations and comments provided by users on the generated proposed images, and is information used to improve the AI ​​model.

[1180] "Generative AI model" refers to an AI algorithm that uses AI technology to generate new images based on user requests and a database.

[1181] A "prompt sentence" refers to a sentence for inputting a specific generation request to the system, which includes an instruction for generating a proposed image.

[1182] This invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[1183] The server collects image data based on past cases and tags specific features. The collected image data is then automatically tagged with specific features such as color, material, furniture type, and layout using an AI algorithm installed on the server. This AI algorithm uses AI frameworks such as TensorFlow and PyTorch. Once tagged, the images are stored in a database on the server.

[1184] The user inputs specific office layout elements into a request form using a terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be specified. When the user presses the submit button on the request form, the request data is sent to the server.

[1185] The server analyzes the request received from the user and searches for relevant image data from a database based on the specified elements. The server then uses the search results and a generative AI model to generate a new proposed image. This generative AI model uses AI technologies such as GAN (generative adversarial network) and VQ-VAE (vector quantization variational autoencoder). The generated proposed image is then sent from the server to the user's device.

[1186] The user can check the sent proposed image on their own device. Then, the user can input evaluation feedback for the proposed image. For example, the user can input comments such as "I like this design" or "I prefer brighter colors." When the user sends the evaluation feedback, the feedback data is sent to the server.

[1187] The server analyzes the feedback received from the user and updates the parameters of the AI ​​model based on the analysis results. This analysis can be performed using data analysis tools such as Python and Pandas. The updated AI model is reflected in the next image generation, enabling the generation of high-quality proposed images that are more suited to the user's requests.

[1188] As a concrete example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and presses the submit button, sending the request data to the server. The server searches past case data for image data containing these elements, and the generative AI model generates a new proposed image. The generated image is sent to the user's device. If the user reviews the image and rates it as "I really like the design," this feedback information is passed on to the server. Based on this feedback, the algorithm for generating proposed images from the next time onwards is improved.

[1189] An example of a prompt might be, "Please suggest an open-plan office design with wooden desks and blue carpet." Based on this prompt, the server can generate an appropriate proposal image.

[1190] As a result, the office furniture proposal process is carried out quickly and accurately, and high-quality proposals are made in response to user requests.

[1191] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1192] Step 1: Data collection and tagging (server side)

[1193] The server collects image data from past case photos and product catalogs. Specifically, it acquires images using web crawling technology and manual uploads. It then uses AI frameworks such as TensorFlow and PyTorch to automatically tag the acquired image data with specific features, such as color, material, furniture type, and layout. It uses raw image data as input, performs feature extraction based on this, and generates tagged image data as output. This data is then stored in a database for efficient search.

[1194] Step 2: Accepting user requests (user / device side)

[1195] The user inputs the desired office layout elements into the request form on the terminal. Specifically, they input elements such as "wooden desks," "blue carpet," and "open layout," and then press the submit button. The input includes a prompt text that the user enters, which is sent from the terminal as an HTTP request to the server. The server receives the request and prepares to analyze it.

[1196] Step 3: Request analysis and image data search (server side)

[1197] The server analyzes the received request and searches the database for relevant image data based on the specified element (prompt sentence). This analysis uses natural language processing technology, and uses SQL queries and Elasticsearch to extract relevant data from the database. The input is the prompt sentence and the database, and by searching based on this, the server obtains relevant image data as the output.

[1198] Step 4: Creating a new image (server side)

[1199] The server uses a generative AI model (such as GAN or VQ-VAE) to generate a new proposed image based on the image data obtained as a search result. Specifically, related image data is used as input, and the AI ​​model analyzes this data to generate a new, high-quality image that includes the desired elements. The search result image data is the input, and the generated proposed image is the output.

[1200] Step 5: Providing generated images (server / device side)

[1201] The server sends the generated proposed image to the user's device. Image data is sent to the user's device as an HTTP response, and the user can view the image. The generated proposed image is the input, and it is obtained as the output to be provided to the user via HTTP.

[1202] Step 6: Evaluation and feedback of proposed images (user / device side)

[1203] The user checks the proposed image on the device and inputs their evaluation feedback, such as "I like the design" or "Brighter colors would be better," and then presses the send button. The user's evaluation feedback is input and is sent to the server.

[1204] Step 7: Analyze feedback and update the AI ​​model (server side)

[1205] The server analyzes the received feedback and updates the parameters of the AI ​​model based on the results. Data analysis tools such as Python and Pandas are used for the analysis, and a dataset is prepared for retraining the model. User feedback is used as input, and the AI ​​model is updated based on this, obtaining an output that will be reflected in subsequent image generation.

[1206] Through these steps, the system can generate high-quality suggested images that quickly and precisely meet user requirements, and improve the AI ​​model in real time based on feedback.

[1207] (Application example 1)

[1208] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1209] The design and layout of modern office environments are becoming increasingly complex, requiring rapid, high-quality proposals that meet user requirements. However, proposing office furniture and layouts is costly and difficult to do efficiently. Viewing physical samples and actual layouts is particularly time-consuming and requires a great deal of trial and error before users are satisfied. Furthermore, conventional systems struggle to effectively utilize user feedback, resulting in the time required to improve the accuracy of the proposed images.

[1210] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1211] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received request, means for receiving feedback from the user for evaluating the generated image, means for updating the AI ​​model based on the feedback, and means for the user to visualize the generated image in real time in a virtual environment. This allows the user to visualize the office layout in a virtual environment in real time, making the proposal process efficient and low-cost. Furthermore, by effectively utilizing user feedback, the accuracy of the AI ​​model can be improved, enabling faster, higher-quality proposals.

[1212] "Image data" is digital data containing visual information, such as photographs and illustrations of office furniture and layout.

[1213] "Tagging" is the process of adding labels to collected image data to identify specific features (e.g., color, material, furniture type, layout, etc.).

[1214] A "generation request" refers to a request made by a user to specify desired office furniture and layout elements and to request the server to generate a new proposed image based on those elements.

[1215] "Feedback" is evaluation information provided by the user regarding the generated proposed image, and includes comments regarding color, layout, design, and the like, for example.

[1216] An "AI model" is a machine learning algorithm or network that uses artificial intelligence technology to analyze data and generate new proposed images.

[1217] "Real-time visualization" is a technology that allows users to instantly see the office layout and furniture placement in a virtual environment.

[1218] A "virtual environment" is a virtual space or screen generated by computer simulation that provides a realistic visual experience despite not being real.

[1219] The present invention relates to a system for generating high-quality proposal images efficiently and at low cost in the office furniture proposal process. This system is operated in cooperation between a server, a terminal, and a user.

[1220] Basic system configuration

[1221] server

[1222] The server has a means for collecting image data based on past cases. This image data includes photographs and illustrations of office furniture and its layout, and is digital data containing visual information. The collected image data is given labels (tagging means) to identify specific features (e.g., color, material, furniture type, layout, etc.).

[1223] The server has a means for receiving a generation request based on elements of an office layout specified by a user. The user inputs the elements using a smartphone, smart glasses, or a head-mounted display, and sends the request to the server.

[1224] The received generation request is analyzed, and related image data is searched for in the database based on the specified elements, and the AI ​​model generates a new proposed image based on that (image generation means). The generated image is sent to the user's device and visualized in real time in a virtual environment.

[1225] The server has a means for receiving user feedback on the generated images, including comments on color, placement, design, etc. This feedback is analyzed and used to update the parameters of the AI ​​model, improving the accuracy of future suggestions (AI model update means).

[1226] Hardware and software used

[1227] Hardware: On the client side, it uses a smartphone, smart glasses, or a head-mounted display. On the server side, it uses a high-performance server or cloud infrastructure.

[1228] Software: Deep learning frameworks such as TensorFlow and PyTorch are used for training the AI ​​model and generating images, and frameworks such as Flask and FastAPI are used for API communication.

[1229] Specific examples

[1230] As a concrete example, consider the case where a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form via a smartphone, smart glasses, or head-mounted display, and sends it to the server. The server searches past case data for material images that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can view it in real time in a virtual environment.

[1231] Next, the user provides feedback on the proposed image, such as "I really like the design." This feedback information is sent to the server, which analyzes it and updates the parameters of the AI ​​model. This improves the accuracy of image generation from the next time onwards, enabling the AI ​​model to respond more quickly and accurately to user requests.

[1232] Prompt Sentence Examples

[1233] Featured elements: wooden desk, blue carpet, open layout

[1234] Feedback: I really like the design

[1235] This system will enable users to quickly and accurately receive suggestions for their desired office layout, and is expected to revolutionize the office furniture suggestion process.

[1236] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1237] Step 1:

[1238] The user specifies the elements of the proposed office layout. Using a smartphone, smart glasses, or a head-mounted display, the user enters elements such as "wooden desks," "blue carpet," and "open layout" into a request form. This input data is then sent to the server.

[1239] Step 2:

[1240] The server analyzes the generation request received from the user. Based on the received element data (e.g., "wooden desk," "blue carpet," "open layout"), the server searches for related image data from past case studies. Here, an AI model is used to identify related materials based on the specified elements.

[1241] Step 3:

[1242] The server generates a proposed image of a new office layout based on the identified material data. It uses an AI model (e.g., a generative artificial network (GAN)) to combine related image data and generate a highly accurate image. The generated image data is then sent to the user's device.

[1243] Step 4:

[1244] The user checks the proposed image displayed on the device. The user visualizes the office layout in real time in a virtual environment using a smartphone, smart glasses, or a head-mounted display. Here, real-time visualization allows the user to see how realistic the proposed image will look.

[1245] Step 5:

[1246] The user provides evaluation feedback for the proposed image. For example, if the user evaluates the design as "I really like it," this feedback information is sent to the server via the terminal.

[1247] Step 6:

[1248] The server analyzes the feedback received from the user. The server adjusts and updates the parameters of the AI ​​model based on the feedback. Specifically, it uses the feedback data to perform re-learning and parameter optimization to improve the accuracy of the AI ​​model. This improves the accuracy of the proposed image generation from the next time onwards, making it more adaptable to user requests.

[1249] Through these steps, users can quickly and accurately receive suggestions for their desired office layout. The server and AI model use feedback to continuously improve, so the suggestion process becomes more accurate over time.

[1250] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1251] The present invention relates to a system that makes the office furniture recommendation process more effective by combining an emotion engine that recognizes the user's emotions. This system is operated in cooperation between four parties: a server, a terminal, a user, and the emotion engine.

[1252] Basic system configuration

[1253] The server has the ability to collect image data based on past cases and tag this data with specific features. The tagged data is stored in a database. The user uses their device to specify the elements to be used in the proposed office layout and sends the request to the server. The server generates new proposed images based on the received request and provides them to the user. The user provides evaluation feedback on the generated images, and the server uses this to update the AI ​​model. Furthermore, an emotion engine analyzes the user's emotions and reflects this data in the imaging process to improve the quality of the proposals.

[1254] Program processing overview

[1255] Data collection and tagging (server side)

[1256] The server collects image data from past proposal photos and product catalogs. This image data is tagged with specific characteristics (e.g., color, material, furniture type, layout, etc.) using pre-defined AI algorithms. Once tagged, the image data is stored in a database and used for subsequent image generation.

[1257] Receiving a generation request (user / device side)

[1258] The user specifies elements for a specific office layout proposal through the terminal. For example, elements such as "wooden desks," "blue carpet," and "open layout" can be considered. These elements are entered into a request form on the terminal and sent to the server.

[1259] Image generation (server side)

[1260] The server analyzes the received request and searches a database for relevant image data based on the specified elements. The AI ​​model generates a new suggested image based on the search results and provides it to the user. The generated image combines the specified elements to achieve a realistic visual representation.

[1261] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[1262] When a user checks the proposed image, the emotion engine analyzes the user's facial expressions and voice using the device's built-in camera and microphone. For example, if the user is smiling or has a dissatisfied expression, emotional data is collected in real time.

[1263] The emotion engine analyzes this data to identify emotions such as happiness, surprise, or confusion. The server then uses this emotion data to adjust the image generation process. For example, if the user responds well to bright colors, the server will use more bright colors in the next suggestions.

[1264] Image confirmation and evaluation (user device side)

[1265] The generated proposed image is sent to the user's device, where the user can review it. The user then provides feedback on the proposed image, including evaluations of specific elements (e.g., "I prefer a slightly brighter color"). The user's emotional state is also included as part of the evaluation.

[1266] Feedback and AI model updates (server side)

[1267] The server receives and analyzes user feedback (including emotion engine data). Based on the analysis results, the parameters of the AI ​​model are updated. This improves the accuracy of image generation from the next time onwards, allowing it to better adapt to user requests.

[1268] Specific examples

[1269] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for image elements that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. The emotion engine recognizes the user's smiling expression through the device's camera and sends it to the server. The server uses this emotion data to adjust its next proposal to increase elements that evoke positive emotions.

[1270] This allows users to quickly and accurately receive proposals for their desired office layout, further revolutionizing the office furniture proposal process.

[1271] The processing flow will be explained below.

[1272] Program processing flow

[1273] Data collection and tagging (server side)

[1274] Step 1:

[1275] The server collects image data from past proposal case photos and product catalogs, typically collected automatically from existing company databases or external sources.

[1276] Step 2:

[1277] The server categorizes the collected image data and stores it in a database. Data is categorized by project name, time period, type, etc.

[1278] Step 3:

[1279] The server uses AI algorithms to tag the stored image data with specific features (e.g., color, material, furniture type, layout, etc.) The tagging is done automatically, and the model is updated regularly.

[1280] Receiving a generation request (user / device side)

[1281] Step 4:

[1282] The user starts up the terminal and accesses the interface of the proposed system.

[1283] Step 5:

[1284] Users input the elements of the office layout they want to propose (color, material, furniture type, layout, etc.) into a form on the interface. For example, "wooden desks," "blue carpet," and "open layout" are examples.

[1285] Step 6:

[1286] The terminal converts the user's input content into structured generated request data and transmits it to the server.

[1287] Image generation (server side)

[1288] Step 7:

[1289] The server analyzes the creation request data received from the terminal and identifies the specified element.

[1290] Step 8:

[1291] The server searches the image data in its database for material that matches the specified elements, extracting the relevant images based on tagged metadata.

[1292] Step 9:

[1293] The server uses the AI ​​model to generate new suggested images from the search results, compositing the images to properly reflect the elements specified by the user.

[1294] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[1295] Step 10:

[1296] When the user checks the proposed image, an emotion engine is activated that uses the device's built-in camera and microphone to collect the user's facial expressions and voice.

[1297] Step 11:

[1298] The emotion engine analyzes the user's facial expression data and voice tone in real time to extract emotional data, such as smiles, furrowed brows, and changes in voice tone.

[1299] Step 12:

[1300] The emotion engine sends the analyzed emotion data to the server, which is then classified into categories such as "happiness," "surprise," and "confusion."

[1301] Step 13:

[1302] The server dynamically adjusts the elements of the proposed images based on the emotional data. For example, if the user responds positively to bright colors, the server will increase the number of bright colors in the next proposed image.

[1303] Image confirmation and evaluation (user device side)

[1304] Step 14:

[1305] The server transmits the generated proposed image to the user's terminal.

[1306] Step 15:

[1307] The terminal displays the received proposed image and provides a confirmation screen to the user.

[1308] Step 16:

[1309] The user checks the proposed image and enters feedback in the "Evaluation" form. For example, the user might write a specific evaluation opinion such as "I like the color of the wooden desk, but I would like the carpet to be lighter."

[1310] Step 17:

[1311] The terminal transmits the user's evaluation feedback to the server, which converts the feedback data into a format that makes it easy to analyze the evaluation content.

[1312] Feedback and AI model updates (server side)

[1313] Step 18:

[1314] The server analyzes the evaluation feedback received from the users, and the analysis results include the user's satisfaction level and improvement requests for each element.

[1315] Step 19:

[1316] The server updates the parameters of the AI ​​model based on the analyzed feedback data, enabling it to make suggestions that better match the user's requests when generating the next image.

[1317] Step 20:

[1318] The server stores the updated AI model and new learning data in a database, ready for use in the next proposal process.

[1319] The above steps will enable quick and accurate office layout proposals that take user emotions into consideration, further revolutionizing the office furniture proposal process.

[1320] Example 2

[1321] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1322] Existing office layout proposal systems often fail to take user emotions into account, resulting in proposals that often fail to satisfy users. Furthermore, if the generated proposed images do not match the user's preferences, the AI ​​model is not updated sufficiently based on the feedback, resulting in a lack of improvement in proposal accuracy in the future.

[1323] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1324] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for collecting emotional data from the user's facial expressions and voice using a terminal, means for analyzing the emotional data and reflecting the emotional data in the generation process, means for receiving feedback from the user to evaluate the generated image, and means for updating the AI ​​model based on the feedback. This enables office layout proposals that take the user's emotions into consideration, thereby improving the accuracy of proposals and user satisfaction.

[1325] "Image data based on past cases" refers to image files collected from past performance data and product catalogs that can be used as reference for office layout proposals.

[1326] A "specific feature" is an identifiable attribute such as color, material, furniture type, layout, etc. in the image data.

[1327] "Tagging" is the process of adding specific features as labels to image data.

[1328] A "generation request" is a request sent to the server based on the elements of the office layout proposed by the user.

[1329] A "new image" is a proposed image generated using an AI model in response to a user's generation request.

[1330] A "terminal" is an electronic device such as a computer, smartphone, or tablet that is operated by a user.

[1331] "Emotion data" refers to information about the user's emotional state analyzed from their facial expressions and voice.

[1332] An "emotion engine" is software that analyzes collected emotional data and identifies the user's emotions.

[1333] "Feedback" refers to user evaluations and opinions regarding the generated proposed image.

[1334] An "AI model" is a mathematical model that uses artificial intelligence technology to learn and make predictions based on input data.

[1335] "Parameters" are internal settings used to adjust the functions and behavior of an AI model.

[1336] "Analysis" is the process of examining collected data and extracting necessary information.

[1337] The present invention relates to a system that recognizes and reflects the user's emotions, thereby making the office furniture recommendation process more effective. This system operates in cooperation with four parties: a server, a terminal, a user, and an emotion engine.

[1338] Basic system configuration

[1339] Server: The server collects image data from past office proposals and product catalogs. This image data is tagged with specific features using an AI algorithm. The tagged data is then stored in a database.

[1340] Terminal: The user specifies elements for a specific office layout proposal through the terminal. For example, elements could be "wooden desks," "blue carpet," and "open layout." The terminal then sends these elements to the server.

[1341] Emotion Engine: When a user reviews a proposed image, the device's built-in camera and microphone are used to collect the user's facial expressions and voice. The emotion engine analyzes this data and identifies the user's emotions. This emotion data is sent to the server and reflected in the image generation process.

[1342] AI model: The server uses the AI ​​model to generate new suggested images based on the request and emotion data received from the user. The generated images are sent to the device and provided to the user. The user reviews the suggested images and provides feedback. The server updates the parameters of the AI ​​model based on this feedback, improving the accuracy of future suggestions.

[1343] Hardware and software used

[1344] Hardware: Servers, databases, user devices (PCs, tablets, smartphones), cameras, microphones

[1345] Software: Image processing libraries (e.g., OpenCV), machine learning models (e.g., TensorFlow), sentiment analysis libraries (e.g., Microsoft Azure Emotion API), cloud storage APIs, RESTful APIs

[1346] Specific examples

[1347] For example, suppose a user specifies "wooden desks," "blue carpet," and "open layout" when proposing a new office layout. The user enters these elements into a request form and sends it to the server. The server searches past case data for image elements that incorporate these elements, and the AI ​​model uses them to generate a new proposed image. The generated image is sent to the user's device, where the user can review it. The emotion engine recognizes the user's smiling expression through the device's camera and sends it to the server. The server uses this emotion data to adjust its next proposal to increase elements that evoke positive emotions.

[1348] Prompt Sentence Examples

[1349] “If a user specifies wooden desks, blue carpet, and an open layout, generate proposed images of office layouts that include these elements, evaluate the user’s reactions using an emotion engine, and reflect them in your next proposal.”

[1350] According to the present invention, a user can quickly and accurately receive a proposal for a desired office layout, and the office furniture proposal process can be carried out more efficiently.

[1351] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1352] Step 1: Data collection and tagging (server side)

[1353] The server collects image data from past office proposal cases and product catalogs. Specifically, it uses a cloud storage API to download image files from multiple data sources. Next, the server uses an image processing library (e.g., OpenCV) and a machine learning model (e.g., TensorFlow) to tag the image data with specific features (e.g., color, material, furniture type, layout, etc.). Through this process, the image data is tagged with feature tags and stored in a database.

[1354] Input: Past case photos, product catalog images

[1355] Output: Image data with feature tags (stored in a database)

[1356] Step 2: Receiving the generation request (user / device side)

[1357] A user specifies elements of a particular office layout using a dedicated application on their device. For example, they input elements such as "wooden desks," "blue carpet," and "open layout" into an input form. The device generates an HTTP request based on the input elements and sends it to the server using a RESTful API.

[1358] Input: User requested elements (wooden desk, blue carpet, open layout)

[1359] Output: HTTP request sent to the server

[1360] Step 3: Image generation (server side)

[1361] The server analyzes the received request data using its analysis engine and extracts the request content. It then generates an SQL query to search the database for image data with the specified characteristics. It then uses an AI model (e.g., GANs - Generative Adversarial Networks) to generate new suggested images from the search results and sends them to the user's device as an HTTP response.

[1362] Input: Request data from the user

[1363] Output: Generated proposal image (sent to device)

[1364] Step 4: Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[1365] When a user reviews a proposed image generated on their device, the device's built-in camera and microphone are used to collect the user's facial expressions and voice. The device then uses an emotion analysis library (e.g., Microsoft Azure Emotion API) to identify the user's emotions and sends that data to the server. The server then adjusts the parameters of the AI ​​model based on this emotion data and reflects it in the image generation process from the next time onwards.

[1366] Input: User's facial expression and voice data

[1367] Output: Analyzed emotion data (sent to server)

[1368] Step 5: Check and evaluate the image (user device side)

[1369] The user reviews the proposed images on their device and provides evaluation feedback. The feedback on the proposed images is entered into an input form and sent from the device to the server as an HTTP request.

[1370] Input: User rating feedback

[1371] Output: Feedback data sent to the server

[1372] Step 6: Feedback and AI model updates (server side)

[1373] The server receives user feedback and emotion data and analyzes it using an analysis engine. Based on the analysis results, it uses a machine learning algorithm (e.g., gradient descent) to readjust the parameters of the AI ​​model and improve the accuracy of subsequent image generation.

[1374] Input: User rating feedback, emotion data

[1375] Output: Updated AI model parameters

[1376] This allows the user to receive more accurate office layout proposals.

[1377] (Application example 2)

[1378] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1379] Conventional content delivery systems do not take into account the user's emotional state when making suggestions, making it difficult to appropriately provide content that the user wants to watch. Furthermore, feedback is based solely on ratings, and cannot respond to real-time emotional changes during viewing, making it impossible to provide optimal content to individual users in a timely manner. The present invention aims to solve these problems and realize more personalized content delivery based on the user's emotions.

[1380] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1381] In this invention, the server includes means for collecting image data based on past cases, means for tagging the collected image data with specific features, means for receiving a generation request based on elements suggested by a user, means for generating a new image based on the received generation request, means for receiving feedback from the user for evaluating the generated image, means for updating the AI ​​model based on the feedback, means for acquiring user emotional data, and means for adjusting the image generation process based on the acquired emotional data. This makes it possible to analyze the user's emotional state in real time and provide optimal content based on that.

[1382] "Past cases" refer to previously recorded data or information, specifically records of content that a user has previously viewed or rated.

[1383] "Image data" refers to digitized visual information, and specifically includes thumbnails and video frames of content.

[1384] "Tagging" is the process of adding specific attributes or characteristics to data, making it easier to search and classify.

[1385] "Suggestion" refers to the system's action of providing content and layout to the user, showing the optimal options based on the user's requests and conditions.

[1386] A "generation request" is a request that includes information that allows a user to request the system to generate content or an image.

[1387] "Image generation" refers to the process of creating a new image based on specified elements, and is done by combining the necessary information from AI models and databases.

[1388] "Feedback" refers to users' evaluations and reactions to the generated content and services, and this data is used to improve the system.

[1389] An "AI model" is an algorithm built using machine learning and deep learning, which is used to analyze data and make new suggestions and generate images.

[1390] "Emotion data" is data that indicates the emotional state of the user, obtained from facial expressions, voice, behavior, and the like.

[1391] "Tuning" refers to changing the parameters of a system or model in order to improve accuracy or performance.

[1392] The present invention is a system that analyzes a user's emotional data and proposes personalized content based on that data. This system is operated through cooperation between four parties: a server, a terminal, a user, and an emotion engine.

[1393] Basic system configuration

[1394] Data collection and tagging (server side)

[1395] The server collects past viewing history and rating data and tags it with specific characteristics, such as the genre of the video viewed, actors, emotional triggers, and ratings, etc. This data is then stored in a database and used for subsequent content suggestions.

[1396] Receiving a generation request (user / device side)

[1397] A user uses a device to request suggestions for specific content, such as "I want to watch funny videos." These elements are entered into a request form on the smartphone and sent to the server.

[1398] Video suggestions (server side)

[1399] The server analyzes the received request and searches a database for relevant video data based on the specified elements. It then uses a generative AI model to generate new suggested videos and provide them to the user. The generated videos combine the specified elements to provide a high-quality visual experience.

[1400] Acquiring and reflecting emotional data (user, device, emotion engine, server side)

[1401] While a user is watching a video, the emotion engine analyzes the user's facial expressions and voice using the device's built-in camera and microphone. For example, if the user is smiling, emotion data is collected in real time. The emotion engine analyzes this data and identifies emotions such as joy or displeasure. The server then adjusts the next content suggestion process based on the emotion data.

[1402] Video review and evaluation (user device side)

[1403] The generated suggested videos are sent to the user's device, where the user watches them. The user then provides evaluation feedback for the suggested videos, including evaluations of specific elements and the user's emotional state.

[1404] Feedback and AI model updates (server side)

[1405] The server receives and analyzes feedback from users (including emotional data). Based on the analysis results, the parameters of the generative AI model are readjusted. This improves the accuracy of video suggestions from the next time onwards, enabling the provision of content that is more suited to the user's preferences.

[1406] Hardware and software used

[1407] Hardware

[1408] Smartphone: Watching videos, using the camera and microphone

[1409] Server: Database, generative AI model

[1410] software

[1411] OpenCV: Smartphone camera control

[1412] dlib: face detection

[1413] EmotionRecognizer: A virtual emotion analysis library

[1414] requests: HTTP request library

[1415] Specific examples

[1416] When a user requests "I want to watch a funny video," the server searches for related comedy videos based on past viewing history and rating data, and generates new suggested videos using a generative AI model. The generated video is sent to the user's smartphone, and the user begins watching it. The smartphone's camera and microphone recognize the user's smile in real time through the emotion engine. As a result, the next suggested video will contain many elements that are likely to make the user smile.

[1417] Example prompt sentence:

[1418] "I'm stressed out today, so I want to watch a relaxing video."

[1419] "Please make a video that will cheer you up."

[1420] "I've been feeling sad lately, so I want to watch a video that will make me feel cheerful."

[1421] Based on these prompts, the system can provide content that best suits the user's emotional state in real time.

[1422] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1423] Step 1:

[1424] The server collects past viewing history and rating data. The specific input is the history of videos watched by users and their rating data, and based on this, tagged images and metadata are stored in a database. The output is a tagged database.

[1425] Data processing involves adding specific characteristics such as genre, actors, emotional triggers, and ratings to the collected data.

[1426] Step 2:

[1427] The user inputs a content suggestion request using a terminal. The specific input is a desired content, such as "I want to watch a funny video," and this is sent to the server. The output is the request information.

[1428] Data processing involves analyzing the user's request and sending the necessary information to the server.

[1429] Step 3:

[1430] The server analyzes the user's request and searches for relevant video data from the database. The specific input is the request content and database information. The output is raw data for the generative AI model.

[1431] Data processing involves searching for related materials based on the specified elements and extracting data.

[1432] Step 4:

[1433] The server generates new suggested videos using a generative AI model. The specific input is the retrieved material data. The output is the newly generated suggested videos.

[1434] Data processing is the process of generating a video based on the material data searched by the AI ​​model.

[1435] Step 5:

[1436] The user watches the generated suggested videos on their device. The specific input is the generated suggested videos. The output is the user's viewing experience and feedback information.

[1437] No data processing is required, and viewing experience and feedback are provided.

[1438] Step 6:

[1439] The device acquires the user's emotional data through a camera and microphone and analyzes it with an emotion engine. The specific input is the user's facial expression and voice. The output is the analyzed emotional data.

[1440] Data processing involves using an emotion engine to analyze emotion data and identify the emotional state.

[1441] Step 7:

[1442] The terminal transmits the user's emotional data to the server. The specific input is the emotional data analyzed by the emotion engine. The output is the emotional data transmitted to the server.

[1443] No data processing is required and the data is transmitted.

[1444] Step 8:

[1445] The server analyzes the user's feedback and emotion data and updates the AI ​​model. The specific input is the user's feedback data and emotion data. The output is the updated AI model.

[1446] Data processing involves analyzing feedback data and emotional data and readjusting the parameters of the AI ​​model.

[1447] Step 9:

[1448] The server adjusts the next content suggestion process. The specific inputs are the updated AI model and the user's emotional data. The output is a new content suggestion process.

[1449] In terms of data processing, the next proposal will be made more personalized based on the newly obtained data.

[1450] Through these steps, it becomes possible to analyze user emotions in real time and provide optimal content based on that.

[1451] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1452] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1453] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1454] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1455] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1456] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1457] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1458] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1459] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1460] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1461] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1462] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1463] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1464] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1465] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1466] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1467] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1468] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1469] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1470] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1471] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1472] The following is further disclosed regarding the above embodiment.

[1473] (Claim 1)

[1474] A means for collecting image data based on past cases;

[1475] means for tagging the collected image data with specific features;

[1476] means for receiving a generation request based on user-proposed elements;

[1477] means for generating a new image based on the received generation request;

[1478] means for receiving feedback from a user for evaluating the generated image;

[1479] A means to update the AI ​​model based on feedback; and

[1480] A system including:

[1481] (Claim 2)

[1482] The system according to claim 1, wherein the means for generating a new image based on a generation request searches for related materials from an image database based on specified elements and combines them to generate a highly accurate image.

[1483] (Claim 3)

[1484] 2. The system of claim 1, wherein the means for updating the AI ​​model based on feedback analyzes the evaluation feedback received from the user and readjusts parameters of the AI ​​model.

[1485] "Example 1"

[1486] (Claim 1)

[1487] A means for collecting image data based on past cases;

[1488] means for tagging the collected image data with specific features;

[1489] means for receiving a generation request based on user-proposed elements;

[1490] means for generating a new image based on the received generation request;

[1491] means for providing the generated image to a user;

[1492] means for receiving feedback from a user;

[1493] A means of analyzing the feedback and updating the parameters of the AI ​​model;

[1494] A means for generating the next image using the updated AI model; and

[1495] A system including:

[1496] (Claim 2)

[1497] The system of claim 1, wherein the means for generating a new image based on a generation request searches for related material from an image database based on specified elements and generates a highly accurate image using a generative AI model.

[1498] (Claim 3)

[1499] 2. The system of claim 1, wherein the means for updating the AI ​​model based on feedback analyzes the evaluation feedback received from the user, readjusts the parameters of the AI ​​model, and reflects the feedback in the next image generation.

[1500] "Application Example 1"

[1501] (Claim 1)

[1502] A means for collecting image data based on past cases;

[1503] means for tagging the collected image data with specific features;

[1504] means for receiving a generation request based on user-proposed elements;

[1505] means for generating a new image based on the received generation request;

[1506] means for receiving feedback from a user for evaluating the generated image;

[1507] A means to update the AI ​​model based on feedback; and

[1508] a means for a user to visualize the generated image in the virtual environment in real time;

[1509] A system including:

[1510] (Claim 2)

[1511] The system according to claim 1, wherein the means for generating a new image based on a generation request searches for related materials from an image database based on specified elements and combines them to generate a highly accurate image.

[1512] (Claim 3)

[1513] 2. The system of claim 1, wherein the means for updating the AI ​​model based on feedback analyzes the evaluation feedback received from the user and readjusts parameters of the AI ​​model.

[1514] "Example 2: Combining Emotion Engines"

[1515] (Claim 1)

[1516] A means for collecting image data based on past cases;

[1517] means for tagging the collected image data with specific features;

[1518] means for receiving a generation request based on user-proposed elements;

[1519] means for generating a new image based on the received generation request;

[1520] A means for collecting emotion data from a user's facial expressions and voice using a terminal;

[1521] A means of analyzing emotion data and reflecting it in the generation process;

[1522] means for receiving feedback from a user for evaluating the generated image;

[1523] A means to update the AI ​​model based on feedback; and

[1524] A system including:

[1525] (Claim 2)

[1526] The system according to claim 1, wherein the means for generating a new image based on a generation request searches for related materials from an image database based on specified elements and combines them to generate a highly accurate image.

[1527] (Claim 3)

[1528] 2. The system of claim 1, wherein the means for updating the AI ​​model based on feedback analyzes the evaluation feedback and emotion data received from the user and readjusts parameters of the AI ​​model.

[1529] "Application example 2 when combining emotion engines"

[1530] (Claim 1)

[1531] A means for collecting image data based on past cases;

[1532] means for tagging the collected image data with specific features;

[1533] means for receiving a generation request based on user-proposed elements;

[1534] means for generating a new image based on the received generation request;

[1535] means for receiving feedback from a user for evaluating the generated image;

[1536] A means to update the AI ​​model based on feedback; and

[1537] A means for acquiring user emotion data;

[1538] a means for adjusting the image generation process based on the acquired emotion data;

[1539] A system including:

[1540] (Claim 2)

[1541] The system according to claim 1, wherein the means for generating a new image based on a generation request searches for related materials from an image database based on specified elements and combines them to generate a highly accurate image.

[1542] (Claim 3)

[1543] 2. The system of claim 1, wherein the means for updating the AI ​​model based on feedback analyzes the evaluation feedback received from the user and readjusts parameters of the AI ​​model. [Explanation of symbols]

[1544] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting image data based on past cases; means for tagging the collected image data with specific features; means for receiving a generation request based on user-proposed elements; means for generating a new image based on the received generation request; means for receiving feedback from a user for evaluating the generated image; A means to update the AI ​​model based on feedback; and A system including:

2. 2. The system according to claim 1, wherein the means for generating a new image based on a generation request searches an image database for related materials based on specified elements and combines them to generate a highly accurate image.

3. 2. The system of claim 1, wherein the means for updating the AI ​​model based on feedback analyzes the evaluation feedback received from the user and readjusts parameters of the AI ​​model.

Images