System

Abstract

A system is provided.SOLUTION: A system, comprising: means for collecting preference information and interior style information of a user; means for analyzing detailed preferences of the user through analysis of images, videos, or music; means for generating and updating a preference model of the user; means for retrieving an optimal artwork from a database based on the preference model; and means for presenting the retrieved artwork to the user and explaining why the artwork is optimal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, it is difficult to find a work that suits your tastes and preferences from the wide variety of artworks available. Furthermore, users with no knowledge of art lack the criteria and information to select the artwork that best suits them. This creates the challenge of limiting the opportunities for the works of many contemporary artists to receive proper evaluation. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for collecting user preference information and interior style information, a means for analyzing the user's detailed preferences through image, video, or music analysis, a means for generating and updating a user preference model, a means for searching a database for suitable artworks based on the preference model, and a means for presenting the searched artworks to the user and explaining why the artworks are suitable. This system allows users to easily find the contemporary artworks that best suit them and at the same time deepen their understanding of art. It is also expected to contribute to revitalizing the art market and improving the reputation of contemporary artists.

[0006] "Preference information" is information that indicates a user's preferences, interests, and concerns.

[0007] "Interior style information" is information about the interior design of the user's home, office, etc.

[0008] "User detailed preferences" is information that details a user's specific preferences and interests.

[0009] A "preference model" is a data model for analysis and prediction created based on user preference information and related data.

[0010] "Artwork" refers to a work of art such as a painting, sculpture, or photograph.

[0011] A "database" is a system for systematically managing multiple artworks and related information.

[0012] "Search" is the act of finding desired information from a database based on specific criteria.

[0013] "Image analysis" is the technique of extracting specific features and information from image data.

[0014] "Music analysis" is a technology that extracts attributes such as genre and theme from music data.

[0015] "Video analysis" is a technology that extracts features such as scenes, themes, and color tones from video data.

[0016] "Feedback Information" refers to evaluations and opinions provided by users based on the results of their selections or evaluations. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is mainly composed of a server and a terminal, and users access the system through the terminal.

[0039] When the system starts, the user logs in to the system using a terminal. On first use, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal sends this entered data to the server, which then stores the received data in a database.

[0040] After collecting user information, the server generates initial survey questions for the user based on this information. The device interactively displays the questions to the user, who answers them. The device sends the answers to the server, which analyzes them and updates the user's preference data. The device then prompts the user to upload images, videos, and music. When the user uploads these media, the device sends the data to the server.

[0041] The server runs uploaded images through image analysis algorithms to evaluate color and style. Similarly, it analyzes music and video data to extract genres and themes. It combines these analysis results to generate and update a detailed user preference model. The server then searches its database for the most suitable artwork based on the preference model. It selects multiple candidates from the search results and generates a comment explaining why each artwork is the best fit for the user.

[0042] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[0043] As a concrete example, the following scenario can be considered.

[0044] Example: Art suggestions based on hobbies and interior styles

[0045] 1. User: Enters the system and types in "art for my living room."

[0046] 2. Terminal: The user is prompted to "upload a photo of your living room."

[0047] 3. User: Uploads a photo of their living room.

[0048] 4. Server: Analyzes the uploaded photos and evaluates their color and style (modern, classic, etc.).

[0049] 5. Device: Display a question to the user: "What are your favorite colors or elements (abstract paintings, small objects, specific themes, etc.)?"

[0050] 6. User: "I like the color blue and I like abstract art."

[0051] 7. Server: Based on the collected information, search the database for abstract paintings with blue as the main color.

[0052] 8. Server: Select multiple candidates from the search results and write a description of why each piece of art would be suitable for the user's living room.

[0053] 9. Terminal: Presents the user with a list of candidate artworks and descriptions.

[0054] 10. User: Selects the artwork they like and proceeds to checkout.

[0055] In this way, the system suggests the most suitable artworks based on the user's tastes and interior style, encouraging them to purchase art.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] A user logs into the system using a terminal.

[0059] Step 2:

[0060] The device displays a user registration form when used for the first time.

[0061] Step 3:

[0062] The user enters basic information such as name, email address, hobbies, interior style, and budget.

[0063] Step 4:

[0064] The terminal transmits the input data to the server.

[0065] Step 5:

[0066] The server stores the received data in a database.

[0067] Step 6:

[0068] The server generates questions for the user based on the results of the initial survey.

[0069] Step 7:

[0070] The device interactively asks the user questions (e.g., "What is your favorite genre of art?").

[0071] Step 8:

[0072] The user enters the answer.

[0073] Step 9:

[0074] The device sends the response to the server.

[0075] Step 10:

[0076] The server analyzes the responses and updates the user's preference data.

[0077] Step 11:

[0078] The device prompts the user to upload images, videos, and music.

[0079] Step 12:

[0080] Users upload photos, videos, and music.

[0081] Step 13:

[0082] The terminal transmits the uploaded data to the server.

[0083] Step 14:

[0084] The server runs the uploaded photos through image analysis algorithms to evaluate color and style.

[0085] Step 15:

[0086] The server analyzes the music and video data and extracts genres and themes.

[0087] Step 16:

[0088] The server integrates the analysis results to generate and update a detailed preference model of the user.

[0089] Step 17:

[0090] The server searches the database for the most suitable artwork based on the preference model.

[0091] Step 18:

[0092] The server generates a list of artworks as search results.

[0093] Step 19:

[0094] The server generates comments explaining why each artwork is suitable for the user.

[0095] Step 20:

[0096] The terminal displays the generated list of artworks and explanatory comments to the user.

[0097] Step 21:

[0098] The user reviews the proposed artworks and selects the one they like best.

[0099] Step 22:

[0100] Your device will display detailed information and a link to purchase the selected title.

[0101] Step 23:

[0102] The user completes a purchase or provides feedback about the work.

[0103] Step 24:

[0104] The device sends feedback or purchase information to the server.

[0105] Step 25:

[0106] The server receives the feedback and updates the user's preference data.

[0107] This series of steps suggests artworks that best suit the user's tastes and interior style, deepening their understanding of art and encouraging them to make a purchase.

[0108] Example 1

[0109] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0110] Today's consumers have a wide variety of tastes and interior styles, making it difficult to select artworks that suit them. Finding artworks that match a user's tastes and interior styles takes a lot of time and effort. Furthermore, because user preferences change over time, it is necessary to constantly make suggestions that reflect the latest preferences. Given this situation, there is a need for a method to analyze detailed user preference data and efficiently suggest the most suitable artworks.

[0111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0112] In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for the most suitable artwork based on the preference model, means for presenting the searched artwork to the user and explaining why the artwork is the most suitable, and means for collecting user feedback or purchase information and updating the preference model, thereby making it possible to suggest the most suitable artwork based on the user's tastes and interior style.

[0113] "Preference information" is information about a user's personal preferences, such as their preferred colors, styles, genres, and themes.

[0114] "Interior style information" refers to information about the decoration and design of a user's home or living space, such as modern, classic, minimalist, and other styles.

[0115] "Analysis means" refers to algorithms and software for analyzing data such as images, videos, and music, and is a means for analyzing user preferences in detail.

[0116] A "preference model" is a data model constructed based on collected and analyzed user preference information and interior style information, and reflects the user's preferences.

[0117] A "database" is a database management system for storing information about users and artworks, and is a means of searching and storing information.

[0118] "Search tools" refers to algorithms or software that search the database for suitable artworks based on preference models.

[0119] "Presentation means" refers to the means by which the artworks and descriptions of the search results are displayed to the user, typically via a user interface.

[0120] "Feedback Information" refers to the opinions and ratings that users provide about selected artworks and suggestions, which the system uses to update its preference model.

[0121] "Purchase Information" means information relating to the actual purchase of a selected artwork by a User, including transaction details and history.

[0122] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is primarily composed of a server and a terminal, and users access the system through the terminal. Specific embodiments for implementing the present invention are described below.

[0123] First, the user logs in to the system using a terminal. When using for the first time, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal then sends this entered data to the server. The server then stores the received data in a database. The database uses a general database management system (DBMS), and can be, for example, a database using SQL.

[0124] After collecting user information, the server generates initial survey questions for the user based on this information. The initial survey is intended to collect more detailed user preference data. The device displays the questions to the user in an interactive format, and the user answers them. The answers are sent from the device to the server, which analyzes them and updates the user preference data. The server's analysis function can use Python-based AI models (such as TensorFlow and OpenCV).

[0125] The device then prompts the user to upload images, videos, and music. Once the user uploads these media files, the device sends the data to a server. The server runs the uploaded images through image analysis algorithms (TensorFlow and OpenCV) to evaluate color and style. Similarly, the server analyzes music and video data to extract genres and themes. This analysis uses natural language processing techniques (e.g., SpaCy and the BERT model).

[0126] The server combines these analysis results to generate and update a detailed user preference model. Based on the preference model, the server searches the database for the most suitable artworks. The algorithm used for the search can use recommendation techniques (e.g., collaborative filtering or content-based filtering). From the search results, multiple candidates are selected and a comment is generated explaining why each artwork is the best fit for the user.

[0127] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[0128] As a concrete example, below we show the steps for proposing art based on hobbies and interior style.

[0129] 1. User: Enters the system and types in "art for my living room."

[0130] 2. Terminal: The user is prompted to "upload a photo of your living room."

[0131] 3. User: Uploads a photo of their living room.

[0132] 4. Server: Analyzes the uploaded photos and evaluates their color and style (modern, classic, etc.).

[0133] 5. Device: Display a question to the user: "What are your favorite colors or elements (abstract paintings, small objects, specific themes, etc.)?"

[0134] 6. User: "I like the color blue and I like abstract art."

[0135] 7. Server: Based on the collected information, search the database for abstract paintings with blue as the main color.

[0136] 8. Server: Select multiple candidates from the search results and write a description of why each piece of art would be suitable for the user's living room.

[0137] 9. Terminal: Presents the user with a list of candidate artworks and descriptions.

[0138] 10. User: Select the artwork they like and proceed to checkout.

[0139] An example of a prompt sentence to input to the generative AI model is as follows:

[0140] "Collect information about the user's tastes and interior style, and suggest suitable artworks. For example, analyze the user's interior photos, favorite colors and themes, and generate a list of suitable artworks based on the results."

[0141] In this way, the system aims to increase user satisfaction by suggesting the most suitable artwork based on the user's tastes and interior style.

[0142] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0143] Step 1:

[0144] User login

[0145] Input: The user enters their email address and password into the device.

[0146] Processing: The terminal sends the input data to the server, which checks it against a database and performs authentication.

[0147] Output: If authentication is successful, the server sends an authentication success message to the terminal and the user is redirected to the main screen.

[0148] What happens: The server runs an SQL query to match the user data in the database, and if there is a match, the authentication is passed.

[0149] Step 2:

[0150] Enter and submit initial registration information

[0151] Input: The user enters basic information into the device, such as their name, email address, hobbies, interior style, and budget.

[0152] Processing: The terminal sends the input data to the server, which stores the data in a database.

[0153] Output: The server sends a registration complete message to the terminal.

[0154] Specific operation: The server structures the received data and stores it in a database using SQL.

[0155] Step 3:

[0156] Generate your initial survey and collect responses

[0157] Input: The server receives the user's basic information and generates an initial survey.

[0158] Processing: The initial survey questions are sent to the device, which displays them to the user, who answers them.

[0159] Output: Sends the user's answer data to the server, which analyzes the received answer data and updates the preference data.

[0160] Specific operation: The server dynamically generates a questionnaire template based on basic information, then analyzes it using an NLP model and updates preference data.

[0161] Step 4:

[0162] Uploading and parsing media files

[0163] Input: A user uploads media files such as images, videos, and music.

[0164] Processing: The device sends the media file to the server, which uses image analysis algorithms (e.g., TensorFlow or OpenCV) to evaluate color and style. Similarly, music and video data can be analyzed to extract genres and themes.

[0165] Output: The server generates the analysis result data.

[0166] Specific operation: The server performs hue analysis on the received image data using OpenCV, and performs genre analysis on music files using the BERT model.

[0167] Step 5:

[0168] Creating and updating user preference models

[0169] Input: The server receives basic information, survey responses, and media analysis results.

[0170] Processing: Integrating this data to generate and update a model of user preferences.

[0171] Output: Save the latest preference model data to the database.

[0172] What it does: The server uses statistical models and machine learning algorithms to integrate multiple data sources and generate a model of user preferences.

[0173] Step 6:

[0174] Proposal of the best artwork

[0175] Input: The server bases its current preference model on it.

[0176] Processing: Search the database for the most suitable artwork and select multiple candidates using recommendation algorithms (e.g. collaborative filtering, content-based filtering).

[0177] Output: The server generates a list of suggested artworks for the user, along with explanatory comments for each.

[0178] Specific operation: The server uses an SQL query to search the database for artworks that match the preference model and generates comments to present them to the user.

[0179] Step 7:

[0180] View and purchase artwork

[0181] Input: User reviews a list of suggested artworks.

[0182] Action: Select the artwork you like and view more information and a link to purchase.

[0183] Output: If the customer wishes to proceed with the purchase, proceed to the purchase screen. If not, display the feedback input screen.

[0184] Specific operation: The terminal dynamically generates and displays the purchase procedure screen or feedback input screen according to the user's selection.

[0185] Step 8:

[0186] Collecting feedback and updating preference data

[0187] Input: Users enter feedback on artwork.

[0188] Processing: The device sends the feedback data to the server, which analyzes the feedback and updates the user's preference data.

[0189] Output: Save the updated preference data to the database.

[0190] Specific operation: The server analyzes the feedback data using natural language processing technology and saves the new data in the database to update the preference model again.

[0191] (Application example 1)

[0192] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0193] Conventional art recommendation systems make recommendations based solely on the user's preferences and interior style information, which tends to neglect the in-store experience. This makes it difficult to match the art pieces the user actually sees in the store, which reduces convenience when making purchasing decisions. Furthermore, since real-time recommendations based on the user's behavior and preferences in the store are not made, there is also the problem of customer satisfaction not being sufficiently improved.

[0194] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0195] In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for suitable artworks based on the preference model, means for presenting the searched artworks to the user and explaining why the artworks are suitable, and means for the user to take an image of an object in a store and suggest suitable artworks for the store. This allows for quick and accurate matching with artworks the user has actually seen in the physical store, making it possible to suggest suitable artworks that suit the user's preferences in real time.

[0196] "User preference information" refers to data about the user's preferences, such as their preferred art style, color, and theme.

[0197] "Interior style information" is information about the design and decoration style that the user prefers in their living space.

[0198] "Image, video or music analysis" refers to the process of analyzing media data provided by users to understand their content and characteristics.

[0199] "Detailed user preferences" are the result of a detailed analysis of the specific types of artwork that users prefer.

[0200] A "preference model" is a data model that represents a user's preferences and is generated based on the user's preference information and interior style information.

[0201] "Database" refers to a data storage system for storing and managing user information and artwork information.

[0202] A "searching means" is a function that extracts information from a database based on specific conditions.

[0203] An "art work" is an artistic creation such as a painting, sculpture, or photograph.

[0204] "Means for users to take pictures of objects within the store" refers to a function that allows users to take pictures of artwork within the store using a smartphone, camera, etc.

[0205] "Means of suggestion" is a function that recommends appropriate art works to users based on collected data and analysis results.

[0206] The system of this invention, "ArtShop Assistant," is a system that suggests artworks that match the user's taste in real time in a physical store. The main components of this system are a server and a terminal (smartphone) used by the user.

[0207] In the initial setup, the user logs in to the system using a terminal and registers as a user. The user's basic information, such as name, email address, hobbies, interior style, and budget, is sent from the terminal to the server and stored in a database. The server generates an initial questionnaire and displays the questions to the user in an interactive format via the terminal. The user enters their answers, and the data is sent back to the server, updating the user's preference data.

[0208] When a user searches for an artwork in a store, they use their device to take a picture of the object. This image data is sent to a server, which then uses an image analysis algorithm (e.g., OpenCV) to analyze the image's color and style. The analysis results are then reflected back into the user's preference model, updating it as detailed preference data.

[0209] The server searches the database for the most suitable artwork based on the user's preference model and the evaluation results from image analysis, and evaluates the search results using a generative AI model. This generative AI model recommends appropriate artworks in real time based on the user's prompt. Typical examples of prompts are as follows:

[0210] User: I like modern interiors with darker tones. I'm looking for abstract art with a predominant blue color.

[0211] AI prompt: Suggest artwork that takes into account the user's interior style (dark tones, modern) and preferences (blue, abstract art).

[0212] The device displays a list of artworks sent from the server and the reasons why each artwork is the best fit. The user can select their favorite artwork from the list and proceed to the purchase process. Even if they do not purchase, they can enter feedback, which is sent to the server and used to further update the user's preference data.

[0213] This allows users to find artworks that match their tastes in real time in a physical store. The main hardware used is a smartphone and a server, and the software includes a web application framework using Flask on the server side, image analysis libraries such as OpenCV, and generative AI models. This improves the user's in-store experience and makes it easier to recommend and purchase the most suitable artworks.

[0214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0215] Step 1:

[0216] When a user logs in for the first time, the device displays a user registration form. The user enters basic information such as name, email address, hobbies, interior style, and budget, and the device sends this information to the server, which stores the received data in a database.

[0217] Input: User's basic information (name, email address, hobbies, interior style, budget)

[0218] Output: User information stored in the database

[0219] Step 2:

[0220] The server generates an initial questionnaire based on the user information and displays the questions interactively to the user through the terminal. The user answers the questions, and the terminal sends the answers to the server. The server analyzes the answers and updates the user's preference data.

[0221] Input: User's survey response

[0222] Output: Updated user preference data

[0223] Step 3:

[0224] When a user searches for an artwork in a store, they take a picture of the object with their device. This image data is sent to a server, which then uses image analysis algorithms (such as OpenCV) to analyze the image's color and style. The analysis results are reflected in the user's preference data.

[0225] Input: A photographed image of the object

[0226] Output: Analyzed image color and style data

[0227] Step 4:

[0228] The server searches the database for the most suitable artwork based on the user's preference model and the results of image analysis, and then uses a generative AI model to recommend suitable artworks in real time, matching them with the user's prompts.

[0229] Input: User preference model, image analysis results

[0230] Output: A list of the best artworks

[0231] Step 5:

[0232] The server generates a list of artworks and the reasons why they are the best, and sends it to the device. The device displays this information to the user, who can then select the artwork they like from the list.

[0233] Input: A list of the best artworks, and why they're the best.

[0234] Output: A list of artworks displayed to the user

[0235] Step 6:

[0236] When the user selects a piece of art they like, the device sends the selection to the server, which then generates a checkout link and sends it to the device. The user can click the link to complete the purchase. Even if the user does not purchase, they can still enter feedback, which is sent from the device to the server, which then updates the preference data.

[0237] Input: User's artwork selection or feedback

[0238] Output: Checkout page link, updated preference data

[0239] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0240] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is mainly composed of a server, a terminal, and an emotion engine, and users access the system through their terminal.

[0241] When the system starts, the user logs in to the system using a terminal. On first use, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal sends this entered data to the server, which then stores the received data in a database.

[0242] After collecting user information, the server generates initial survey questions for the user based on this information. The device displays the questions interactively to the user, who answers them. The answers are sent from the device to the server, which analyzes them and updates the user's preference data. The device then prompts the user to input images, videos, music, and emotional states through an emotion engine. After the user uploads these media and provides emotional information, the device sends the data to the server.

[0243] The server runs uploaded images through image analysis algorithms to evaluate color and style. Similarly, it analyzes music and video data to extract genres and themes. The emotion engine analyzes the user's current emotional state based on facial expressions and voice, and sends that information to the server.

[0244] The server integrates these analysis results to generate and update a detailed user preference model. Based on the preference model, the server searches the database for the most suitable artworks. From the search results, the server selects multiple candidates and generates comments explaining why each artwork is the best fit for the user.

[0245] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[0246] As a concrete example, the following scenario can be considered.

[0247] Example: Art suggestions based on tastes and emotional states

[0248] 1. User: Access the system and select "Art to watch when you want to relax."

[0249] 2. On the device, the user is prompted to upload photos and music to relax with.

[0250] 3. User: Upload photos of the music you like to listen to when relaxing or the interior design.

[0251] 4. Server: Analyzes the uploaded data and evaluates the color and mood of the music.

[0252] 5. Emotion engine: Analyzes the user's current emotional state and sends it to the server as a "want to relax" state.

[0253] 6. Server: Using the collected information, search the database for suitable artworks for relaxation.

[0254] 7. Server: Select multiple candidates from the search results and write a description of each artwork explaining why it is suitable for the user.

[0255] 8. Terminal: Presents the user with a list of candidate artworks and descriptions.

[0256] 9. User: Selects the artwork they like and proceeds to checkout.

[0257] In this way, the system takes into account the user's tastes, interior style, and emotional state to suggest the most suitable artwork, deepening their understanding of art and encouraging them to make purchases.

[0258] The processing flow will be explained below.

[0259] Step 1:

[0260] A user logs into the system using a terminal.

[0261] Step 2:

[0262] The device displays a user registration form when used for the first time.

[0263] Step 3:

[0264] The user enters basic information such as name, email address, hobbies, interior style, and budget.

[0265] Step 4:

[0266] The terminal transmits the input data to the server.

[0267] Step 5:

[0268] The server stores the received data in a database.

[0269] Step 6:

[0270] The server generates questions for the user based on the results of the initial survey.

[0271] Step 7:

[0272] The device interactively asks the user questions (e.g., "What is your favorite genre of art?").

[0273] Step 8:

[0274] The user enters the answer.

[0275] Step 9:

[0276] The device sends the response to the server.

[0277] Step 10:

[0278] The server analyzes the responses and updates the user's preference data.

[0279] Step 11:

[0280] The device prompts the user to input images, videos, music, and emotional states using an emotion engine.

[0281] Step 12:

[0282] Users upload photos, videos, music and emotional information.

[0283] Step 13:

[0284] The terminal transmits the uploaded data to the server.

[0285] Step 14:

[0286] The server runs the uploaded photos through image analysis algorithms to evaluate color and style.

[0287] Step 15:

[0288] The server analyzes the music and video data and extracts genres and themes.

[0289] Step 16:

[0290] The emotion engine analyzes the user's emotional state from their facial expressions and voice and sends the data to the server.

[0291] Step 17:

[0292] The server integrates the analysis results (images, music, videos, emotions) to generate and update a detailed preference model of the user.

[0293] Step 18:

[0294] The server searches the database for the most suitable artwork based on the preference model.

[0295] Step 19:

[0296] The server generates a list of artworks as search results.

[0297] Step 20:

[0298] The server generates comments explaining why each artwork is suitable for the user.

[0299] Step 21:

[0300] The terminal displays the generated list of artworks and explanatory comments to the user.

[0301] Step 22:

[0302] The user reviews the proposed artworks and selects the one they like best.

[0303] Step 23:

[0304] Your device will display detailed information and a link to purchase the selected title.

[0305] Step 24:

[0306] The user completes a purchase or provides feedback about the work.

[0307] Step 25:

[0308] The device sends feedback or purchase information to the server.

[0309] Step 26:

[0310] The server receives the feedback and updates the user's preference data.

[0311] This series of steps suggests the most suitable artworks based on the user's tastes, interior style, and even emotional state, helping them better understand art and encourage them to make purchases.

[0312] Example 2

[0313] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] Conventional art suggestion systems have the problem of not being able to suggest the most suitable artworks because they do not fully consider the user's preferences or emotional state. In addition, they lack a system to reflect changes in user preferences, making it difficult to respond to individual needs.

[0315] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through a questionnaire, means for analyzing image, music, or video media data, means for analyzing the user's emotional state, means for integrating the analysis results to generate and update a user preference model, means for searching a database for an optimal visual artwork based on the preference model, and means for presenting the searched visual artwork to the user and explaining why the artwork is optimal. This makes it possible to suggest optimal artworks that take the user's preferences and emotional state into consideration.

[0316] "User preference information and interior style information" is information related to the user's hobbies and preferences, and the interior design style of the user's home, office, etc.

[0317] A "survey" is a set of questions distributed to users to understand their detailed hobbies and preferences.

[0318] "Image, music or video media data" refers to image files, music files and video files uploaded by users for relaxation or related to their hobbies.

[0319] "User's emotional state" is information that indicates the user's current emotional or psychological state.

[0320] "Means for integrating analysis results to generate and update a user preference model" refers to the process of analyzing collected data to create and update an individual preference model that reflects the user's tastes and emotional state.

[0321] The "means for retrieving optimal visual artworks from a database based on a preference model" is a process for retrieving optimal artworks from a database using a user's preference model.

[0322] "A means of presenting visual artworks resulting from a search result to a user and explaining why each artwork is the most appropriate" is a process of presenting artworks retrieved from a database to a user and explaining why each artwork is the most appropriate for the user.

[0323] "Feedback Information" refers to the opinions and ratings you provide regarding the artworks you select or purchase.

[0324] The present invention is a system that performs detailed analysis based on a user's preference information and interior style information to suggest the most suitable artwork. The system mainly includes a server, a terminal, and an emotion engine. The user accesses the system through the terminal.

[0325] System configuration

[0326] 1. Means of collecting user preference information and interior style information

[0327] Device: When using for the first time, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget.

[0328] Terminal: Sends the entered information to the server.

[0329] Server: Receives user information and stores it in a database.

[0330] 2. A means of analyzing detailed user preferences through surveys

[0331] Server: Generates initial survey questions based on user information.

[0332] Terminal: Questions are presented to the user interactively, and the user answers them.

[0333] Terminal: Sends survey responses to the server.

[0334] Server: Analyzes the received survey responses and updates the user preference data.

[0335] 3. Means for analyzing image, music or video media data

[0336] Terminal: Provides an interface for users to upload photos of their favorite music and interior design when relaxing.

[0337] User: Upload media data such as images, music, and videos.

[0338] Terminal: Sends the uploaded data to the server.

[0339] Server: Analyzes the received image data using an image analysis algorithm (e.g., OpenCV).

[0340] Server: Analyzes music and video data using a library (e.g., LIBROSA) to extract genres and themes.

[0341] 4. A means of analyzing the user's emotional state

[0342] Emotion engine: Analyzes the user's current emotional state from their facial expressions and voice, and sends that information to the server.

[0343] 5. Means of integrating the analysis results to generate and update user preference models

[0344] Server: Integrates the analysis results of media data and emotional information to generate and update a detailed user preference model.

[0345] 6. A method for searching a database for suitable visual artworks based on a preference model

[0346] Server: Search for visual artworks from a database using the preference model (e.g., using Elasticsearch).

[0347] 7. A way to present users with visual art results and explain why they are the best.

[0348] Server: Selects candidate works from the search results and uses a generative AI model to generate comments explaining why each one is the best.

[0349] Terminal: Displays a list of artworks and explanatory comments to the user.

[0350] Specific examples

[0351] Example: Art suggestions based on tastes and emotional states

[0352] 1. User: Access the system and select "Art to watch when you want to relax."

[0353] 2. On the device, the user is prompted to upload photos and music to relax with.

[0354] 3. User: Upload photos of the music you like to listen to when relaxing or the interior design.

[0355] 4. Server: Analyzes the uploaded data and evaluates the color and musical atmosphere (using OpenCV for image data and LIBROSA for music data).

[0356] 5. Emotion engine: Analyzes the user's current emotional state and sends it to the server as a "want to relax" state.

[0357] 6. Server: Based on the collected information, search the database for suitable artworks for relaxation (using Elasticsearch).

[0358] 7. Server: Selects multiple candidates from the search results and uses a generative AI model to compile a description of why each artwork is suitable for the user.

[0359] 8. Terminal: Presents the user with a list of candidate artworks and descriptions.

[0360] 9. User: Selects the artwork they like and proceeds to checkout.

[0361] Prompt Sentence Examples

[0362] plain text

[0363] Describe art suggestions based on user preferences, interior style, and emotional state. Consider the following:

[0364] User basic information (name, email address, hobbies, interior style, budget)

[0365] User-uploaded media data (images, music, videos)

[0366] Emotion engine analysis results

[0367] Pick the best piece of art and explain why it's the best fit for your users.

[0368] In this way, the system of the present invention can effectively suggest the most suitable artworks by reflecting the preferences and emotional state of each user in detail.

[0369] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0370] Step 1:

[0371] A user accesses the system and logs in by entering their name and password in the login form. When using the system for the first time, the terminal displays a user registration form, and the user enters basic information such as their name, email address, hobbies, interior style, and budget. The entered information is sent from the terminal to the server.

[0372] Input: Basic information such as name, email address, hobbies, interior style, budget, etc.

[0373] Output: Basic information about the user sent to the server

[0374] Step 2:

[0375] The server receives the user's basic information sent from the device and stores it in a database, which is used for subsequent analysis.

[0376] Input: Basic user information sent from the device

[0377] Output: Basic user information stored in the database

[0378] Step 3:

[0379] The server generates initial survey questions based on the user information, and sends the generated questions to the terminal, which then displays the survey questions to the user in an interactive format. The user answers the questions.

[0380] Input: Basic user information stored in the database

[0381] Output: Generated initial survey questions

[0382] Step 4:

[0383] The user answers the questionnaire and enters the answers into the terminal, which then sends the user's answers to the server.

[0384] Input: User's survey response

[0385] Output: Survey responses sent to the server

[0386] Step 5:

[0387] The server analyzes the received survey responses and updates the user's preference data using a text analysis algorithm.

[0388] Input: User's survey response

[0389] Output: Updated user preference data

[0390] Step 6:

[0391] The device provides a user with an interface that prompts the user to upload media data such as images, videos, and music, and input emotional information. The user uploads relevant media data and inputs emotional information, and these data are transmitted from the device to the server.

[0392] Input: User-uploaded media data and emotional information

[0393] Output: Media data and emotion information sent to the server

[0394] Step 7:

[0395] The server analyzes the received image data using an image analysis algorithm (e.g., OpenCV). Music and video data is analyzed using an analysis library (e.g., LIBROSA) to extract genres and themes. The emotion engine also analyzes the current emotional state from facial expressions and voice, and this information is sent to the server.

[0396] Input: image data, music data, video data, emotional information

[0397] Output: Analysis results (color, style, genre, theme, emotional state)

[0398] Step 8:

[0399] The server integrates the analysis results and generates and updates a detailed user preference model using data fusion techniques.

[0400] Input: Analysis results

[0401] Output: User preference model

[0402] Step 9:

[0403] The server searches the database for the most suitable visual artworks based on the preference model, using a database search engine (e.g., Elasticsearch).

[0404] Input: User preference model

[0405] Output: Searched visual artwork candidates

[0406] Step 10:

[0407] The server selects candidate artworks from the search results and uses a generative AI model to generate comments explaining why each is the best fit. The generated comments and list of artworks are then sent to the device.

[0408] Input: Search results for visual artworks

[0409] Output: Generated explanatory comments and artwork list

[0410] Step 11:

[0411] The device displays the generated explanatory comments and a list of artworks to the user.

[0412] Input: Generated descriptive comments and artwork list

[0413] Output: A list of artworks and explanatory comments displayed to the user

[0414] Step 12:

[0415] The user reviews the proposed artworks and selects the one they like. Detailed information and a link to purchase the selected artwork are displayed. If the user does not purchase, they can enter their opinion through a feedback form. The feedback or purchase information is sent from the device to the server.

[0416] Input: User's work selection, feedback or purchase information

[0417] Output: Feedback or purchase information sent to the server

[0418] Step 13:

[0419] The server receives the feedback or purchase information and updates the user's preference data again.

[0420] Input: Feedback or Purchase Information

[0421] Output: Updated user preference data

[0422] (Application example 2)

[0423] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0424] In conventional logistics centers, real-time analysis of worker efficiency and emotional state is not adequately performed, and optimal work instructions are not provided according to the state of each worker, resulting in problems such as reduced work efficiency and mistakes due to overwork.In addition, dynamic task adjustment based on employee feedback is not performed, making it difficult to optimize the entire operation.

[0425] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for suitable artworks based on the preference model, means for presenting the searched artworks to the user and explaining why the artworks are suitable, and means for analyzing the user's work efficiency and emotional state using a smart device and providing optimal task instructions. This makes it possible to dynamically adjust work instructions in real time and provide optimal instructions according to the status of each worker.

[0426] "User preference information" is information that reflects the user's hobbies, preferences, and interests.

[0427] "Interior style information" is information about the interior decoration of a user's home or office.

[0428] "Image, video or music analysis" refers to the analysis of these media data to assess a user's preferences and emotional state.

[0429] "Detailed user preferences" refers to information about a user's specific hobbies and preferences, extracted from the analyzed data.

[0430] A "user preference model" is a data model that reflects a user's preferences and is generated based on the user's collected preference information.

[0431] The "optimal artwork" is the artwork selected based on the user's preference model and determined to be most suitable for the user.

[0432] A "smart device" is an advanced device that can connect to the Internet and is equipped with various sensors, and examples include smart glasses.

[0433] "Work efficiency" refers to the efficiency and productivity of workers when performing their work.

[0434] "Emotional state" refers to the user's mental state or emotions, including stress, fatigue, joy, etc.

[0435] "Optimal task instructions" are instructions that are most suitable for each task, based on the user's work efficiency and emotional state.

[0436] The system of the present invention analyzes user preference information and emotional states to provide optimal task instructions in order to improve work efficiency and the emotional state of workers in logistics centers. The system mainly consists of a server, terminals, and smart devices (e.g., smart glasses).

[0437] First, the worker (user) wears a smart device and accesses the system through that device. At this time, the user's preference information and interior style information are collected and sent to the server. The server stores this information in a database.

[0438] The server then analyzes media data such as images, videos, and music to assess the user's detailed preferences. This analysis is performed using software libraries such as OpenCV and NLTK. To analyze the user's emotional state, sensors on the smart device are used to analyze facial expressions and voice. OpenCV is used for facial expression analysis, and NLTK, a natural language processing technology, is used for voice analysis.

[0439] Based on the analysis of the user's detailed preference data and emotional state, the server generates and updates a user preference model. Based on the preference model, the server searches for the most suitable artwork from the database and presents it to the user. At the same time, the server analyzes the user's work efficiency and emotional state in real time via the smart device and provides appropriate task instructions.

[0440] As a concrete example, consider the case where an employee at a logistics center uses smart glasses. If the smart glasses detect high stress levels in the employee through facial expression analysis while they are working, the server sends an instruction message to the smart glasses saying, "Perform light work." After the work is completed, feedback from the employee is collected and stored in a database. This feedback is reflected in the next task instructions.

[0441] An example of a prompt sentence is as follows:

[0442] "Build a system that analyzes the emotional state of logistics center employees and adjusts their workload via smart glasses. Specifically, this would include a function that uses facial expression recognition and emotion analysis to determine stress and fatigue, and provides appropriate work instructions in real time."

[0443] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0444] Step 1:

[0445] The user puts on a smart device (smart glasses) and logs into the system. After logging in, the first time they use the system, they are prompted to enter their preferences (hobbies, interior style, budget, etc.). The information entered by the user is sent from the device to the server and stored in a database.

[0446] Input: User preferences (hobbies, interior style, budget, etc.)

[0447] Output: User preferences stored in a database

[0448] Step 2:

[0449] The server generates an initial questionnaire based on the user's preference information and interior style information, and sends the questions to the terminal. The terminal displays the questions to the user in an interactive format and receives the answers.

[0450] Input: User preference information and interior style information

[0451] Output: User's survey responses

[0452] Step 3:

[0453] The user's response data is sent from the device to a server and analyzed to build detailed user preference data. The user also allows the camera for facial expression analysis and microphone for voice analysis to collect emotional state data.

[0454] Input: User survey responses and emotional data

[0455] Output: Detailed preference and sentiment data

[0456] Step 4:

[0457] The server uses OpenCV and NLTK libraries for facial expression and speech analysis, which analyzes camera images and audio samples to assess the user's emotional state (e.g., stress level).

[0458] Input: Camera images and audio samples

[0459] Output: Parsed emotion data (e.g., stress level)

[0460] Step 5:

[0461] Based on the analysis results, the server generates and updates the user's preference model and stores it in the database.Then, based on the preference model, the server searches the database for the most suitable artworks and recommended tasks based on the questionnaire.

[0462] Input: Parsed emotion data and detailed preference data

[0463] Output: Updated preference model and best artwork or recommended work

[0464] Step 6:

[0465] The server then provides the user with real-time task instructions through the smart glasses, tailored to their emotional state. For example, if high stress is detected, instructions for lighter tasks will be displayed.

[0466] Input: Emotional state and preference model

[0467] Output: Task instructions displayed on smart glasses

[0468] Step 7:

[0469] After completing the task, the device collects feedback from the user and sends it to the server, which uses this information to further update the preference model and reflect it in the next task instructions.

[0470] Input: User feedback

[0471] Output: Updated preference model and next task instructions

[0472] In this way, the system of the present invention can provide optimal work instructions in real time based on the user's preference information and emotional state.

[0473] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0474] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0475] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0476] [Second embodiment]

[0477] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0478] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0479] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0480] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0481] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0482] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0483] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0484] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0485] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0486] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0487] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0488] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0489] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is mainly composed of a server and a terminal, and users access the system through the terminal.

[0490] When the system starts, the user logs in to the system using a terminal. On first use, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal sends this entered data to the server, which then stores the received data in a database.

[0491] After collecting user information, the server generates initial survey questions for the user based on this information. The device interactively displays the questions to the user, who answers them. The device sends the answers to the server, which analyzes them and updates the user's preference data. The device then prompts the user to upload images, videos, and music. When the user uploads these media, the device sends the data to the server.

[0492] The server runs uploaded images through image analysis algorithms to evaluate color and style. Similarly, it analyzes music and video data to extract genres and themes. It combines these analysis results to generate and update a detailed user preference model. The server then searches its database for the most suitable artwork based on the preference model. It selects multiple candidates from the search results and generates a comment explaining why each artwork is the best fit for the user.

[0493] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[0494] As a concrete example, the following scenario can be considered.

[0495] Example: Art suggestions based on hobbies and interior styles

[0496] 1. User: Enters the system and types in "art for my living room."

[0497] 2. Terminal: The user is prompted to "upload a photo of your living room."

[0498] 3. User: Uploads a photo of their living room.

[0499] 4. Server: Analyzes the uploaded photos and evaluates their color and style (modern, classic, etc.).

[0500] 5. Device: Display a question to the user: "What are your favorite colors or elements (abstract paintings, small objects, specific themes, etc.)?"

[0501] 6. User: "I like the color blue and I like abstract art."

[0502] 7. Server: Based on the collected information, search the database for abstract paintings with blue as the main color.

[0503] 8. Server: Select multiple candidates from the search results and write a description of why each piece of art would be suitable for the user's living room.

[0504] 9. Terminal: Presents the user with a list of candidate artworks and descriptions.

[0505] 10. User: Selects the artwork they like and proceeds to checkout.

[0506] In this way, the system suggests the most suitable artworks based on the user's tastes and interior style, encouraging them to purchase art.

[0507] The processing flow will be explained below.

[0508] Step 1:

[0509] A user logs into the system using a terminal.

[0510] Step 2:

[0511] The device displays a user registration form when used for the first time.

[0512] Step 3:

[0513] The user enters basic information such as name, email address, hobbies, interior style, and budget.

[0514] Step 4:

[0515] The terminal transmits the input data to the server.

[0516] Step 5:

[0517] The server stores the received data in a database.

[0518] Step 6:

[0519] The server generates questions for the user based on the results of the initial survey.

[0520] Step 7:

[0521] The device interactively asks the user questions (e.g., "What is your favorite genre of art?").

[0522] Step 8:

[0523] The user enters the answer.

[0524] Step 9:

[0525] The device sends the response to the server.

[0526] Step 10:

[0527] The server analyzes the responses and updates the user's preference data.

[0528] Step 11:

[0529] The device prompts the user to upload images, videos, and music.

[0530] Step 12:

[0531] Users upload photos, videos, and music.

[0532] Step 13:

[0533] The terminal transmits the uploaded data to the server.

[0534] Step 14:

[0535] The server runs the uploaded photos through image analysis algorithms to evaluate color and style.

[0536] Step 15:

[0537] The server analyzes the music and video data and extracts genres and themes.

[0538] Step 16:

[0539] The server integrates the analysis results to generate and update a detailed preference model of the user.

[0540] Step 17:

[0541] The server searches the database for the most suitable artwork based on the preference model.

[0542] Step 18:

[0543] The server generates a list of artworks as search results.

[0544] Step 19:

[0545] The server generates comments explaining why each artwork is suitable for the user.

[0546] Step 20:

[0547] The terminal displays the generated list of artworks and explanatory comments to the user.

[0548] Step 21:

[0549] The user reviews the proposed artworks and selects the one they like best.

[0550] Step 22:

[0551] Your device will display detailed information and a link to purchase the selected title.

[0552] Step 23:

[0553] The user completes a purchase or provides feedback about the work.

[0554] Step 24:

[0555] The device sends feedback or purchase information to the server.

[0556] Step 25:

[0557] The server receives the feedback and updates the user's preference data.

[0558] This series of steps suggests artworks that best suit the user's tastes and interior style, deepening their understanding of art and encouraging them to make a purchase.

[0559] Example 1

[0560] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0561] Today's consumers have a wide variety of tastes and interior styles, making it difficult to select artworks that suit them. Finding artworks that match a user's tastes and interior styles takes a lot of time and effort. Furthermore, because user preferences change over time, it is necessary to constantly make suggestions that reflect the latest preferences. Given this situation, there is a need for a method to analyze detailed user preference data and efficiently suggest the most suitable artworks.

[0562] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0563] In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for the most suitable artwork based on the preference model, means for presenting the searched artwork to the user and explaining why the artwork is the most suitable, and means for collecting user feedback or purchase information and updating the preference model, thereby making it possible to suggest the most suitable artwork based on the user's tastes and interior style.

[0564] "Preference information" is information about a user's personal preferences, such as their preferred colors, styles, genres, and themes.

[0565] "Interior style information" refers to information about the decoration and design of a user's home or living space, such as modern, classic, minimalist, and other styles.

[0566] "Analysis means" refers to algorithms and software for analyzing data such as images, videos, and music, and is a means for analyzing user preferences in detail.

[0567] A "preference model" is a data model constructed based on collected and analyzed user preference information and interior style information, and reflects the user's preferences.

[0568] A "database" is a database management system for storing information about users and artworks, and is a means of searching and storing information.

[0569] "Search tools" refers to algorithms or software that search the database for suitable artworks based on preference models.

[0570] "Presentation means" refers to the means by which the artworks and descriptions of the search results are displayed to the user, typically via a user interface.

[0571] "Feedback Information" refers to the opinions and ratings that users provide about selected artworks and suggestions, which the system uses to update its preference model.

[0572] "Purchase Information" means information relating to the actual purchase of a selected artwork by a User, including transaction details and history.

[0573] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is primarily composed of a server and a terminal, and users access the system through the terminal. Specific embodiments for implementing the present invention are described below.

[0574] First, the user logs in to the system using a terminal. When using for the first time, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal then sends this entered data to the server. The server then stores the received data in a database. The database uses a general database management system (DBMS), and can be, for example, a database using SQL.

[0575] After collecting user information, the server generates initial survey questions for the user based on this information. The initial survey is intended to collect more detailed user preference data. The device displays the questions to the user in an interactive format, and the user answers them. The answers are sent from the device to the server, which analyzes them and updates the user preference data. The server's analysis function can use Python-based AI models (such as TensorFlow and OpenCV).

[0576] The device then prompts the user to upload images, videos, and music. Once the user uploads these media files, the device sends the data to a server. The server runs the uploaded images through image analysis algorithms (TensorFlow and OpenCV) to evaluate color and style. Similarly, the server analyzes music and video data to extract genres and themes. This analysis uses natural language processing techniques (e.g., SpaCy and the BERT model).

[0577] The server combines these analysis results to generate and update a detailed user preference model. Based on the preference model, the server searches the database for the most suitable artworks. The algorithm used for the search can use recommendation techniques (e.g., collaborative filtering or content-based filtering). From the search results, multiple candidates are selected and a comment is generated explaining why each artwork is the best fit for the user.

[0578] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[0579] As a concrete example, below we show the steps for proposing art based on hobbies and interior style.

[0580] 1. User: Enters the system and types in "art for my living room."

[0581] 2. Terminal: The user is prompted to "upload a photo of your living room."

[0582] 3. User: Uploads a photo of their living room.

[0583] 4. Server: Analyzes the uploaded photos and evaluates their color and style (modern, classic, etc.).

[0584] 5. Device: Display a question to the user: "What are your favorite colors or elements (abstract paintings, small objects, specific themes, etc.)?"

[0585] 6. User: "I like the color blue and I like abstract art."

[0586] 7. Server: Based on the collected information, search the database for abstract paintings with blue as the main color.

[0587] 8. Server: Select multiple candidates from the search results and write a description of why each piece of art would be suitable for the user's living room.

[0588] 9. Terminal: Presents the user with a list of candidate artworks and descriptions.

[0589] 10. User: Select the artwork they like and proceed to checkout.

[0590] An example of a prompt sentence to input to the generative AI model is as follows:

[0591] "Collect information about the user's tastes and interior style, and suggest suitable artworks. For example, analyze the user's interior photos, favorite colors and themes, and generate a list of suitable artworks based on the results."

[0592] In this way, the system aims to increase user satisfaction by suggesting the most suitable artwork based on the user's tastes and interior style.

[0593] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0594] Step 1:

[0595] User login

[0596] Input: The user enters their email address and password into the device.

[0597] Processing: The terminal sends the input data to the server, which checks it against a database and performs authentication.

[0598] Output: If authentication is successful, the server sends an authentication success message to the terminal and the user is redirected to the main screen.

[0599] What happens: The server runs an SQL query to match the user data in the database, and if there is a match, the authentication is passed.

[0600] Step 2:

[0601] Enter and submit initial registration information

[0602] Input: The user enters basic information into the device, such as their name, email address, hobbies, interior style, and budget.

[0603] Processing: The terminal sends the input data to the server, which stores the data in a database.

[0604] Output: The server sends a registration complete message to the terminal.

[0605] Specific operation: The server structures the received data and stores it in a database using SQL.

[0606] Step 3:

[0607] Generate your initial survey and collect responses

[0608] Input: The server receives the user's basic information and generates an initial survey.

[0609] Processing: The initial survey questions are sent to the device, which displays them to the user, who answers them.

[0610] Output: Sends the user's answer data to the server, which analyzes the received answer data and updates the preference data.

[0611] Specific operation: The server dynamically generates a questionnaire template based on basic information, then analyzes it using an NLP model and updates preference data.

[0612] Step 4:

[0613] Uploading and parsing media files

[0614] Input: A user uploads media files such as images, videos, and music.

[0615] Processing: The device sends the media file to the server, which uses image analysis algorithms (e.g., TensorFlow or OpenCV) to evaluate color and style. Similarly, music and video data can be analyzed to extract genres and themes.

[0616] Output: The server generates the analysis result data.

[0617] Specific operation: The server performs hue analysis on the received image data using OpenCV, and performs genre analysis on music files using the BERT model.

[0618] Step 5:

[0619] Creating and updating user preference models

[0620] Input: The server receives basic information, survey responses, and media analysis results.

[0621] Processing: Integrating this data to generate and update a model of user preferences.

[0622] Output: Save the latest preference model data to the database.

[0623] What it does: The server uses statistical models and machine learning algorithms to integrate multiple data sources and generate a model of user preferences.

[0624] Step 6:

[0625] Proposal of the best artwork

[0626] Input: The server bases its current preference model on it.

[0627] Processing: Search the database for the most suitable artwork and select multiple candidates using recommendation algorithms (e.g. collaborative filtering, content-based filtering).

[0628] Output: The server generates a list of suggested artworks for the user, along with explanatory comments for each.

[0629] Specific operation: The server uses an SQL query to search the database for artworks that match the preference model and generates comments to present them to the user.

[0630] Step 7:

[0631] View and purchase artwork

[0632] Input: User reviews a list of suggested artworks.

[0633] Action: Select the artwork you like and view more information and a link to purchase.

[0634] Output: If the customer wishes to proceed with the purchase, proceed to the purchase screen. If not, display the feedback input screen.

[0635] Specific operation: The terminal dynamically generates and displays the purchase procedure screen or feedback input screen according to the user's selection.

[0636] Step 8:

[0637] Collecting feedback and updating preference data

[0638] Input: Users enter feedback on artwork.

[0639] Processing: The device sends the feedback data to the server, which analyzes the feedback and updates the user's preference data.

[0640] Output: Save the updated preference data to the database.

[0641] Specific operation: The server analyzes the feedback data using natural language processing technology and saves the new data in the database to update the preference model again.

[0642] (Application example 1)

[0643] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0644] Conventional art recommendation systems make recommendations based solely on the user's preferences and interior style information, which tends to neglect the in-store experience. This makes it difficult to match the art pieces the user actually sees in the store, which reduces convenience when making purchasing decisions. Furthermore, since real-time recommendations based on the user's behavior and preferences in the store are not made, there is also the problem of customer satisfaction not being sufficiently improved.

[0645] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0646] In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for suitable artworks based on the preference model, means for presenting the searched artworks to the user and explaining why the artworks are suitable, and means for the user to take an image of an object in a store and suggest suitable artworks for the store. This allows for quick and accurate matching with artworks the user has actually seen in the physical store, making it possible to suggest suitable artworks that suit the user's preferences in real time.

[0647] "User preference information" refers to data about the user's preferences, such as their preferred art style, color, and theme.

[0648] "Interior style information" is information about the design and decoration style that the user prefers in their living space.

[0649] "Image, video or music analysis" refers to the process of analyzing media data provided by users to understand their content and characteristics.

[0650] "Detailed user preferences" are the result of a detailed analysis of the specific types of artwork that users prefer.

[0651] A "preference model" is a data model that represents a user's preferences and is generated based on the user's preference information and interior style information.

[0652] "Database" refers to a data storage system for storing and managing user information and artwork information.

[0653] A "searching means" is a function that extracts information from a database based on specific conditions.

[0654] An "art work" is an artistic creation such as a painting, sculpture, or photograph.

[0655] "Means for users to take pictures of objects within the store" refers to a function that allows users to take pictures of artwork within the store using a smartphone, camera, etc.

[0656] "Means of suggestion" is a function that recommends appropriate art works to users based on collected data and analysis results.

[0657] The system of this invention, "ArtShop Assistant," is a system that suggests artworks that match the user's taste in real time in a physical store. The main components of this system are a server and a terminal (smartphone) used by the user.

[0658] In the initial setup, the user logs in to the system using a terminal and registers as a user. The user's basic information, such as name, email address, hobbies, interior style, and budget, is sent from the terminal to the server and stored in a database. The server generates an initial questionnaire and displays the questions to the user in an interactive format via the terminal. The user enters their answers, and the data is sent back to the server, updating the user's preference data.

[0659] When a user searches for an artwork in a store, they use their device to take a picture of the object. This image data is sent to a server, which then uses an image analysis algorithm (e.g., OpenCV) to analyze the image's color and style. The analysis results are then reflected back into the user's preference model, updating it as detailed preference data.

[0660] The server searches the database for the most suitable artwork based on the user's preference model and the evaluation results from image analysis, and evaluates the search results using a generative AI model. This generative AI model recommends appropriate artworks in real time based on the user's prompt. Typical examples of prompts are as follows:

[0661] User: I like modern interiors with darker tones. I'm looking for abstract art with a predominant blue color.

[0662] AI prompt: Suggest artwork that takes into account the user's interior style (dark tones, modern) and preferences (blue, abstract art).

[0663] The device displays a list of artworks sent from the server and the reasons why each artwork is the best fit. The user can select their favorite artwork from the list and proceed to the purchase process. Even if they do not purchase, they can enter feedback, which is sent to the server and used to further update the user's preference data.

[0664] This allows users to find artworks that match their tastes in real time in a physical store. The main hardware used is a smartphone and a server, and the software includes a web application framework using Flask on the server side, image analysis libraries such as OpenCV, and generative AI models. This improves the user's in-store experience and makes it easier to recommend and purchase the most suitable artworks.

[0665] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0666] Step 1:

[0667] When a user logs in for the first time, the device displays a user registration form. The user enters basic information such as name, email address, hobbies, interior style, and budget, and the device sends this information to the server, which stores the received data in a database.

[0668] Input: User's basic information (name, email address, hobbies, interior style, budget)

[0669] Output: User information stored in the database

[0670] Step 2:

[0671] The server generates an initial questionnaire based on the user information and displays the questions interactively to the user through the terminal. The user answers the questions, and the terminal sends the answers to the server. The server analyzes the answers and updates the user's preference data.

[0672] Input: User's survey response

[0673] Output: Updated user preference data

[0674] Step 3:

[0675] When a user searches for an artwork in a store, they take a picture of the object with their device. This image data is sent to a server, which then uses image analysis algorithms (such as OpenCV) to analyze the image's color and style. The analysis results are reflected in the user's preference data.

[0676] Input: A photographed image of the object

[0677] Output: Analyzed image color and style data

[0678] Step 4:

[0679] The server searches the database for the most suitable artwork based on the user's preference model and the results of image analysis, and then uses a generative AI model to recommend suitable artworks in real time, matching them with the user's prompts.

[0680] Input: User preference model, image analysis results

[0681] Output: A list of the best artworks

[0682] Step 5:

[0683] The server generates a list of artworks and the reasons why they are the best, and sends it to the device. The device displays this information to the user, who can then select the artwork they like from the list.

[0684] Input: A list of the best artworks, and why they're the best.

[0685] Output: A list of artworks displayed to the user

[0686] Step 6:

[0687] When the user selects a piece of art they like, the device sends the selection to the server, which then generates a checkout link and sends it to the device. The user can click the link to complete the purchase. Even if the user does not purchase, they can still enter feedback, which is sent from the device to the server, which then updates the preference data.

[0688] Input: User's artwork selection or feedback

[0689] Output: Checkout page link, updated preference data

[0690] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0691] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is mainly composed of a server, a terminal, and an emotion engine, and users access the system through their terminal.

[0692] When the system starts, the user logs in to the system using a terminal. On first use, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal sends this entered data to the server, which then stores the received data in a database.

[0693] After collecting user information, the server generates initial survey questions for the user based on this information. The device displays the questions interactively to the user, who answers them. The answers are sent from the device to the server, which analyzes them and updates the user's preference data. The device then prompts the user to input images, videos, music, and emotional states through an emotion engine. After the user uploads these media and provides emotional information, the device sends the data to the server.

[0694] The server runs uploaded images through image analysis algorithms to evaluate color and style. Similarly, it analyzes music and video data to extract genres and themes. The emotion engine analyzes the user's current emotional state based on facial expressions and voice, and sends that information to the server.

[0695] The server integrates these analysis results to generate and update a detailed user preference model. Based on the preference model, the server searches the database for the most suitable artworks. From the search results, the server selects multiple candidates and generates comments explaining why each artwork is the best fit for the user.

[0696] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[0697] As a concrete example, the following scenario can be considered.

[0698] Example: Art suggestions based on tastes and emotional states

[0699] 1. User: Access the system and select "Art to watch when you want to relax."

[0700] 2. On the device, the user is prompted to upload photos and music to relax with.

[0701] 3. User: Upload photos of the music you like to listen to when relaxing or the interior design.

[0702] 4. Server: Analyzes the uploaded data and evaluates the color and mood of the music.

[0703] 5. Emotion engine: Analyzes the user's current emotional state and sends it to the server as a "want to relax" state.

[0704] 6. Server: Using the collected information, search the database for suitable artworks for relaxation.

[0705] 7. Server: Select multiple candidates from the search results and write a description of each artwork explaining why it is suitable for the user.

[0706] 8. Terminal: Presents the user with a list of candidate artworks and descriptions.

[0707] 9. User: Selects the artwork they like and proceeds to checkout.

[0708] In this way, the system takes into account the user's tastes, interior style, and emotional state to suggest the most suitable artwork, deepening their understanding of art and encouraging them to make purchases.

[0709] The processing flow will be explained below.

[0710] Step 1:

[0711] A user logs into the system using a terminal.

[0712] Step 2:

[0713] The device displays a user registration form when used for the first time.

[0714] Step 3:

[0715] The user enters basic information such as name, email address, hobbies, interior style, and budget.

[0716] Step 4:

[0717] The terminal transmits the input data to the server.

[0718] Step 5:

[0719] The server stores the received data in a database.

[0720] Step 6:

[0721] The server generates questions for the user based on the results of the initial survey.

[0722] Step 7:

[0723] The device interactively asks the user questions (e.g., "What is your favorite genre of art?").

[0724] Step 8:

[0725] The user enters the answer.

[0726] Step 9:

[0727] The device sends the response to the server.

[0728] Step 10:

[0729] The server analyzes the responses and updates the user's preference data.

[0730] Step 11:

[0731] The device prompts the user to input images, videos, music, and emotional states using an emotion engine.

[0732] Step 12:

[0733] Users upload photos, videos, music and emotional information.

[0734] Step 13:

[0735] The terminal transmits the uploaded data to the server.

[0736] Step 14:

[0737] The server runs the uploaded photos through image analysis algorithms to evaluate color and style.

[0738] Step 15:

[0739] The server analyzes the music and video data and extracts genres and themes.

[0740] Step 16:

[0741] The emotion engine analyzes the user's emotional state from their facial expressions and voice and sends the data to the server.

[0742] Step 17:

[0743] The server integrates the analysis results (images, music, videos, emotions) to generate and update a detailed preference model of the user.

[0744] Step 18:

[0745] The server searches the database for the most suitable artwork based on the preference model.

[0746] Step 19:

[0747] The server generates a list of artworks as search results.

[0748] Step 20:

[0749] The server generates comments explaining why each artwork is suitable for the user.

[0750] Step 21:

[0751] The terminal displays the generated list of artworks and explanatory comments to the user.

[0752] Step 22:

[0753] The user reviews the proposed artworks and selects the one they like best.

[0754] Step 23:

[0755] Your device will display detailed information and a link to purchase the selected title.

[0756] Step 24:

[0757] The user completes a purchase or provides feedback about the work.

[0758] Step 25:

[0759] The device sends feedback or purchase information to the server.

[0760] Step 26:

[0761] The server receives the feedback and updates the user's preference data.

[0762] This series of steps suggests the most suitable artworks based on the user's tastes, interior style, and even emotional state, helping them better understand art and encourage them to make purchases.

[0763] Example 2

[0764] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0765] Conventional art suggestion systems have the problem of not being able to suggest the most suitable artworks because they do not fully consider the user's preferences or emotional state. In addition, they lack a system to reflect changes in user preferences, making it difficult to respond to individual needs.

[0766] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through a questionnaire, means for analyzing image, music, or video media data, means for analyzing the user's emotional state, means for integrating the analysis results to generate and update a user preference model, means for searching a database for an optimal visual artwork based on the preference model, and means for presenting the searched visual artwork to the user and explaining why the artwork is optimal. This makes it possible to suggest optimal artworks that take the user's preferences and emotional state into consideration.

[0767] "User preference information and interior style information" is information related to the user's hobbies and preferences, and the interior design style of the user's home, office, etc.

[0768] A "survey" is a set of questions distributed to users to understand their detailed hobbies and preferences.

[0769] "Image, music or video media data" refers to image files, music files and video files uploaded by users for relaxation or related to their hobbies.

[0770] "User's emotional state" is information that indicates the user's current emotional or psychological state.

[0771] "Means for integrating analysis results to generate and update a user preference model" refers to the process of analyzing collected data to create and update an individual preference model that reflects the user's tastes and emotional state.

[0772] The "means for retrieving optimal visual artworks from a database based on a preference model" is a process for retrieving optimal artworks from a database using a user's preference model.

[0773] "A means of presenting visual artworks resulting from a search result to a user and explaining why each artwork is the most appropriate" is a process of presenting artworks retrieved from a database to a user and explaining why each artwork is the most appropriate for the user.

[0774] "Feedback Information" refers to the opinions and ratings you provide regarding the artworks you select or purchase.

[0775] The present invention is a system that performs detailed analysis based on a user's preference information and interior style information to suggest the most suitable artwork. The system mainly includes a server, a terminal, and an emotion engine. The user accesses the system through the terminal.

[0776] System configuration

[0777] 1. Means of collecting user preference information and interior style information

[0778] Device: When using for the first time, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget.

[0779] Terminal: Sends the entered information to the server.

[0780] Server: Receives user information and stores it in a database.

[0781] 2. A means of analyzing detailed user preferences through surveys

[0782] Server: Generates initial survey questions based on user information.

[0783] Terminal: Questions are presented to the user interactively, and the user answers them.

[0784] Terminal: Sends survey responses to the server.

[0785] Server: Analyzes the received survey responses and updates the user preference data.

[0786] 3. Means for analyzing image, music or video media data

[0787] Terminal: Provides an interface for users to upload photos of their favorite music and interior design when relaxing.

[0788] User: Upload media data such as images, music, and videos.

[0789] Terminal: Sends the uploaded data to the server.

[0790] Server: Analyzes the received image data using an image analysis algorithm (e.g., OpenCV).

[0791] Server: Analyzes music and video data using a library (e.g., LIBROSA) to extract genres and themes.

[0792] 4. A means of analyzing the user's emotional state

[0793] Emotion engine: Analyzes the user's current emotional state from their facial expressions and voice, and sends that information to the server.

[0794] 5. Means of integrating the analysis results to generate and update user preference models

[0795] Server: Integrates the analysis results of media data and emotional information to generate and update a detailed user preference model.

[0796] 6. A method for searching a database for suitable visual artworks based on a preference model

[0797] Server: Search for visual artworks from a database using the preference model (e.g., using Elasticsearch).

[0798] 7. A way to present users with visual art results and explain why they are the best.

[0799] Server: Selects candidate works from the search results and uses a generative AI model to generate comments explaining why each one is the best.

[0800] Terminal: Displays a list of artworks and explanatory comments to the user.

[0801] Specific examples

[0802] Example: Art suggestions based on tastes and emotional states

[0803] 1. User: Access the system and select "Art to watch when you want to relax."

[0804] 2. On the device, the user is prompted to upload photos and music to relax with.

[0805] 3. User: Upload photos of the music you like to listen to when relaxing or the interior design.

[0806] 4. Server: Analyzes the uploaded data and evaluates the color and musical atmosphere (using OpenCV for image data and LIBROSA for music data).

[0807] 5. Emotion engine: Analyzes the user's current emotional state and sends it to the server as a "want to relax" state.

[0808] 6. Server: Based on the collected information, search the database for suitable artworks for relaxation (using Elasticsearch).

[0809] 7. Server: Selects multiple candidates from the search results and uses a generative AI model to compile a description of why each artwork is suitable for the user.

[0810] 8. Terminal: Presents the user with a list of candidate artworks and descriptions.

[0811] 9. User: Selects the artwork they like and proceeds to checkout.

[0812] Prompt Sentence Examples

[0813] plain text

[0814] Describe art suggestions based on user preferences, interior style, and emotional state. Consider the following:

[0815] User basic information (name, email address, hobbies, interior style, budget)

[0816] User-uploaded media data (images, music, videos)

[0817] Emotion engine analysis results

[0818] Pick the best piece of art and explain why it's the best fit for your users.

[0819] In this way, the system of the present invention can effectively suggest the most suitable artworks by reflecting the preferences and emotional state of each user in detail.

[0820] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0821] Step 1:

[0822] A user accesses the system and logs in by entering their name and password in the login form. When using the system for the first time, the terminal displays a user registration form, and the user enters basic information such as their name, email address, hobbies, interior style, and budget. The entered information is sent from the terminal to the server.

[0823] Input: Basic information such as name, email address, hobbies, interior style, budget, etc.

[0824] Output: Basic information about the user sent to the server

[0825] Step 2:

[0826] The server receives the user's basic information sent from the device and stores it in a database, which is used for subsequent analysis.

[0827] Input: Basic user information sent from the device

[0828] Output: Basic user information stored in the database

[0829] Step 3:

[0830] The server generates initial survey questions based on the user information, and sends the generated questions to the terminal, which then displays the survey questions to the user in an interactive format. The user answers the questions.

[0831] Input: Basic user information stored in the database

[0832] Output: Generated initial survey questions

[0833] Step 4:

[0834] The user answers the questionnaire and enters the answers into the terminal, which then sends the user's answers to the server.

[0835] Input: User's survey response

[0836] Output: Survey responses sent to the server

[0837] Step 5:

[0838] The server analyzes the received survey responses and updates the user's preference data using a text analysis algorithm.

[0839] Input: User's survey response

[0840] Output: Updated user preference data

[0841] Step 6:

[0842] The device provides a user with an interface that prompts the user to upload media data such as images, videos, and music, and input emotional information. The user uploads relevant media data and inputs emotional information, and these data are transmitted from the device to the server.

[0843] Input: User-uploaded media data and emotional information

[0844] Output: Media data and emotion information sent to the server

[0845] Step 7:

[0846] The server analyzes the received image data using an image analysis algorithm (e.g., OpenCV). Music and video data is analyzed using an analysis library (e.g., LIBROSA) to extract genres and themes. The emotion engine also analyzes the current emotional state from facial expressions and voice, and this information is sent to the server.

[0847] Input: image data, music data, video data, emotional information

[0848] Output: Analysis results (color, style, genre, theme, emotional state)

[0849] Step 8:

[0850] The server integrates the analysis results and generates and updates a detailed user preference model using data fusion techniques.

[0851] Input: Analysis results

[0852] Output: User preference model

[0853] Step 9:

[0854] The server searches the database for the most suitable visual artworks based on the preference model, using a database search engine (e.g., Elasticsearch).

[0855] Input: User preference model

[0856] Output: Searched visual artwork candidates

[0857] Step 10:

[0858] The server selects candidate artworks from the search results and uses a generative AI model to generate comments explaining why each is the best fit. The generated comments and list of artworks are then sent to the device.

[0859] Input: Search results for visual artworks

[0860] Output: Generated explanatory comments and artwork list

[0861] Step 11:

[0862] The device displays the generated explanatory comments and a list of artworks to the user.

[0863] Input: Generated descriptive comments and artwork list

[0864] Output: A list of artworks and explanatory comments displayed to the user

[0865] Step 12:

[0866] The user reviews the proposed artworks and selects the one they like. Detailed information and a link to purchase the selected artwork are displayed. If the user does not purchase, they can enter their opinion through a feedback form. The feedback or purchase information is sent from the device to the server.

[0867] Input: User's work selection, feedback or purchase information

[0868] Output: Feedback or purchase information sent to the server

[0869] Step 13:

[0870] The server receives the feedback or purchase information and updates the user's preference data again.

[0871] Input: Feedback or Purchase Information

[0872] Output: Updated user preference data

[0873] (Application example 2)

[0874] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0875] In conventional logistics centers, real-time analysis of worker efficiency and emotional state is not adequately performed, and optimal work instructions are not provided according to the state of each worker, resulting in problems such as reduced work efficiency and mistakes due to overwork.In addition, dynamic task adjustment based on employee feedback is not performed, making it difficult to optimize the entire operation.

[0876] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for suitable artworks based on the preference model, means for presenting the searched artworks to the user and explaining why the artworks are suitable, and means for analyzing the user's work efficiency and emotional state using a smart device and providing optimal task instructions. This makes it possible to dynamically adjust work instructions in real time and provide optimal instructions according to the status of each worker.

[0877] "User preference information" is information that reflects the user's hobbies, preferences, and interests.

[0878] "Interior style information" is information about the interior decoration of a user's home or office.

[0879] "Image, video or music analysis" refers to the analysis of these media data to assess a user's preferences and emotional state.

[0880] "Detailed user preferences" refers to information about a user's specific hobbies and preferences, extracted from the analyzed data.

[0881] A "user preference model" is a data model that reflects a user's preferences and is generated based on the user's collected preference information.

[0882] The "optimal artwork" is the artwork selected based on the user's preference model and determined to be most suitable for the user.

[0883] A "smart device" is an advanced device that can connect to the Internet and is equipped with various sensors, and examples include smart glasses.

[0884] "Work efficiency" refers to the efficiency and productivity of workers when performing their work.

[0885] "Emotional state" refers to the user's mental state or emotions, including stress, fatigue, joy, etc.

[0886] "Optimal task instructions" are instructions that are most suitable for each task, based on the user's work efficiency and emotional state.

[0887] The system of the present invention analyzes user preference information and emotional states to provide optimal task instructions in order to improve work efficiency and the emotional state of workers in logistics centers. The system mainly consists of a server, terminals, and smart devices (e.g., smart glasses).

[0888] First, the worker (user) wears a smart device and accesses the system through that device. At this time, the user's preference information and interior style information are collected and sent to the server. The server stores this information in a database.

[0889] The server then analyzes media data such as images, videos, and music to assess the user's detailed preferences. This analysis is performed using software libraries such as OpenCV and NLTK. To analyze the user's emotional state, sensors on the smart device are used to analyze facial expressions and voice. OpenCV is used for facial expression analysis, and NLTK, a natural language processing technology, is used for voice analysis.

[0890] Based on the analysis of the user's detailed preference data and emotional state, the server generates and updates a user preference model. Based on the preference model, the server searches for the most suitable artwork from the database and presents it to the user. At the same time, the server analyzes the user's work efficiency and emotional state in real time via the smart device and provides appropriate task instructions.

[0891] As a concrete example, consider the case where an employee at a logistics center uses smart glasses. If the smart glasses detect high stress levels in the employee through facial expression analysis while they are working, the server sends an instruction message to the smart glasses saying, "Perform light work." After the work is completed, feedback from the employee is collected and stored in a database. This feedback is reflected in the next task instructions.

[0892] An example of a prompt sentence is as follows:

[0893] "Build a system that analyzes the emotional state of logistics center employees and adjusts their workload via smart glasses. Specifically, this would include a function that uses facial expression recognition and emotion analysis to determine stress and fatigue, and provide appropriate work instructions in real time."

[0894] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0895] Step 1:

[0896] The user puts on a smart device (smart glasses) and logs into the system. After logging in, the first time they use the system, they are prompted to enter their preferences (hobbies, interior style, budget, etc.). The information entered by the user is sent from the device to the server and stored in a database.

[0897] Input: User preferences (hobbies, interior style, budget, etc.)

[0898] Output: User preferences stored in a database

[0899] Step 2:

[0900] The server generates an initial questionnaire based on the user's preference information and interior style information, and sends the questions to the terminal. The terminal then displays the questions to the user in an interactive format and receives the answers.

[0901] Input: User preference information and interior style information

[0902] Output: User's survey responses

[0903] Step 3:

[0904] The user's response data is sent from the device to a server and analyzed to build detailed user preference data. The user also allows the camera for facial expression analysis and microphone for voice analysis to collect emotional state data.

[0905] Input: User survey responses and emotional data

[0906] Output: Detailed preference and sentiment data

[0907] Step 4:

[0908] The server uses OpenCV and NLTK libraries for facial expression and speech analysis, which analyzes camera images and audio samples to assess the user's emotional state (e.g., stress level).

[0909] Input: Camera images and audio samples

[0910] Output: Parsed emotion data (e.g., stress level)

[0911] Step 5:

[0912] Based on the analysis results, the server generates and updates the user's preference model and stores it in the database.Then, based on the preference model, the server searches the database for the most suitable artworks and recommended tasks based on the questionnaire.

[0913] Input: Parsed emotion data and detailed preference data

[0914] Output: Updated preference model and best artwork or recommended work

[0915] Step 6:

[0916] The server then provides the user with real-time task instructions through the smart glasses, tailored to their emotional state. For example, if high stress is detected, instructions for lighter tasks will be displayed.

[0917] Input: Emotional state and preference model

[0918] Output: Task instructions displayed on smart glasses

[0919] Step 7:

[0920] After completing the task, the device collects feedback from the user and sends it to the server, which uses this information to further update the preference model and reflect it in the next task instructions.

[0921] Input: User feedback

[0922] Output: Updated preference model and next task instructions

[0923] In this way, the system of the present invention can provide optimal work instructions in real time based on the user's preference information and emotional state.

[0924] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0925] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0926] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0927] [Third embodiment]

[0928] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0929] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0930] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0931] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0932] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0933] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0934] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0935] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0936] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0937] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0938] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0939] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0940] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is mainly composed of a server and a terminal, and users access the system through the terminal.

[0941] When the system starts, the user logs in to the system using a terminal. On first use, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal sends this entered data to the server, which then stores the received data in a database.

[0942] After collecting user information, the server generates initial survey questions for the user based on this information. The device interactively displays the questions to the user, who answers them. The device sends the answers to the server, which analyzes them and updates the user's preference data. The device then prompts the user to upload images, videos, and music. When the user uploads these media, the device sends the data to the server.

[0943] The server runs uploaded images through image analysis algorithms to evaluate color and style. Similarly, it analyzes music and video data to extract genres and themes. It combines these analysis results to generate and update a detailed user preference model. The server then searches its database for the most suitable artwork based on the preference model. It selects multiple candidates from the search results and generates a comment explaining why each artwork is the best fit for the user.

[0944] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[0945] As a concrete example, the following scenario can be considered.

[0946] Example: Art suggestions based on hobbies and interior styles

[0947] 1. User: Enters the system and types in "art for my living room."

[0948] 2. Terminal: The user is prompted to "upload a photo of your living room."

[0949] 3. User: Uploads a photo of their living room.

[0950] 4. Server: Analyzes the uploaded photos and evaluates their color and style (modern, classic, etc.).

[0951] 5. Device: Display a question to the user: "What are your favorite colors or elements (abstract paintings, small objects, specific themes, etc.)?"

[0952] 6. User: "I like the color blue and I like abstract art."

[0953] 7. Server: Based on the collected information, search the database for abstract paintings with blue as the main color.

[0954] 8. Server: Select multiple candidates from the search results and write a description of why each piece of art would be suitable for the user's living room.

[0955] 9. Terminal: Presents the user with a list of candidate artworks and descriptions.

[0956] 10. User: Selects the artwork they like and proceeds to checkout.

[0957] In this way, the system suggests the most suitable artworks based on the user's tastes and interior style, encouraging them to purchase art.

[0958] The processing flow will be explained below.

[0959] Step 1:

[0960] A user logs into the system using a terminal.

[0961] Step 2:

[0962] The device displays a user registration form when used for the first time.

[0963] Step 3:

[0964] The user enters basic information such as name, email address, hobbies, interior style, and budget.

[0965] Step 4:

[0966] The terminal transmits the input data to the server.

[0967] Step 5:

[0968] The server stores the received data in a database.

[0969] Step 6:

[0970] The server generates questions for the user based on the results of the initial survey.

[0971] Step 7:

[0972] The device interactively asks the user questions (e.g., "What is your favorite genre of art?").

[0973] Step 8:

[0974] The user enters the answer.

[0975] Step 9:

[0976] The device sends the response to the server.

[0977] Step 10:

[0978] The server analyzes the responses and updates the user's preference data.

[0979] Step 11:

[0980] The device prompts the user to upload images, videos, and music.

[0981] Step 12:

[0982] Users upload photos, videos, and music.

[0983] Step 13:

[0984] The terminal transmits the uploaded data to the server.

[0985] Step 14:

[0986] The server runs the uploaded photos through image analysis algorithms to evaluate color and style.

[0987] Step 15:

[0988] The server analyzes the music and video data and extracts genres and themes.

[0989] Step 16:

[0990] The server integrates the analysis results to generate and update a detailed preference model of the user.

[0991] Step 17:

[0992] The server searches the database for the most suitable artwork based on the preference model.

[0993] Step 18:

[0994] The server generates a list of artworks as search results.

[0995] Step 19:

[0996] The server generates comments explaining why each artwork is suitable for the user.

[0997] Step 20:

[0998] The terminal displays the generated list of artworks and explanatory comments to the user.

[0999] Step 21:

[1000] The user reviews the proposed artworks and selects the one they like best.

[1001] Step 22:

[1002] Your device will display detailed information and a link to purchase the selected title.

[1003] Step 23:

[1004] The user completes a purchase or provides feedback about the work.

[1005] Step 24:

[1006] The device sends feedback or purchase information to the server.

[1007] Step 25:

[1008] The server receives the feedback and updates the user's preference data.

[1009] This series of steps suggests artworks that best suit the user's tastes and interior style, deepening their understanding of art and encouraging them to make a purchase.

[1010] Example 1

[1011] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1012] Today's consumers have a wide variety of tastes and interior styles, making it difficult to select artworks that suit them. Finding artworks that match a user's tastes and interior styles takes a lot of time and effort. Furthermore, because user preferences change over time, it is necessary to constantly make suggestions that reflect the latest preferences. Given this situation, there is a need for a method to analyze detailed user preference data and efficiently suggest the most suitable artworks.

[1013] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1014] In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for the most suitable artwork based on the preference model, means for presenting the searched artwork to the user and explaining why the artwork is the most suitable, and means for collecting user feedback or purchase information and updating the preference model, thereby making it possible to suggest the most suitable artwork based on the user's tastes and interior style.

[1015] "Preference information" is information about a user's personal preferences, such as their preferred colors, styles, genres, and themes.

[1016] "Interior style information" refers to information about the decoration and design of a user's home or living space, such as modern, classic, minimalist, and other styles.

[1017] "Analysis means" refers to algorithms and software for analyzing data such as images, videos, and music, and is a means for analyzing user preferences in detail.

[1018] A "preference model" is a data model constructed based on collected and analyzed user preference information and interior style information, and reflects the user's preferences.

[1019] A "database" is a database management system for storing information about users and artworks, and is a means of searching and storing information.

[1020] "Search tools" refers to algorithms or software that search the database for suitable artworks based on preference models.

[1021] "Presentation means" refers to the means by which the artworks and descriptions of the search results are displayed to the user, typically via a user interface.

[1022] "Feedback Information" refers to the opinions and ratings that users provide about selected artworks and suggestions, which the system uses to update its preference model.

[1023] "Purchase Information" means information relating to the actual purchase of a selected artwork by a User, including transaction details and history.

[1024] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is primarily composed of a server and a terminal, and users access the system through the terminal. Specific embodiments for implementing the present invention are described below.

[1025] First, the user logs in to the system using a terminal. When using for the first time, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal then sends this entered data to the server. The server then stores the received data in a database. The database uses a general database management system (DBMS), and can be, for example, a database using SQL.

[1026] After collecting user information, the server generates initial survey questions for the user based on this information. The initial survey is intended to collect more detailed user preference data. The device displays the questions to the user in an interactive format, and the user answers them. The answers are sent from the device to the server, which analyzes them and updates the user preference data. The server's analysis function can use Python-based AI models (such as TensorFlow and OpenCV).

[1027] The device then prompts the user to upload images, videos, and music. Once the user uploads these media files, the device sends the data to a server. The server runs the uploaded images through image analysis algorithms (TensorFlow and OpenCV) to evaluate color and style. Similarly, the server analyzes music and video data to extract genres and themes. This analysis uses natural language processing techniques (e.g., SpaCy and the BERT model).

[1028] The server combines these analysis results to generate and update a detailed user preference model. Based on the preference model, the server searches the database for the most suitable artworks. The algorithm used for the search can use recommendation techniques (e.g., collaborative filtering or content-based filtering). From the search results, multiple candidates are selected and a comment is generated explaining why each artwork is the best fit for the user.

[1029] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[1030] As a concrete example, below we show the steps for proposing art based on hobbies and interior style.

[1031] 1. User: Enters the system and types in "art for my living room."

[1032] 2. Terminal: The user is prompted to "upload a photo of your living room."

[1033] 3. User: Uploads a photo of their living room.

[1034] 4. Server: Analyzes the uploaded photos and evaluates their color and style (modern, classic, etc.).

[1035] 5. Device: Display a question to the user: "What are your favorite colors or elements (abstract paintings, small objects, specific themes, etc.)?"

[1036] 6. User: "I like the color blue and I like abstract art."

[1037] 7. Server: Based on the collected information, search the database for abstract paintings with blue as the main color.

[1038] 8. Server: Select multiple candidates from the search results and write a description of why each piece of art would be suitable for the user's living room.

[1039] 9. Terminal: Presents the user with a list of candidate artworks and descriptions.

[1040] 10. User: Select the artwork they like and proceed to checkout.

[1041] An example of a prompt sentence to input to the generative AI model is as follows:

[1042] "Collect information about the user's tastes and interior style, and suggest suitable artworks. For example, analyze the user's interior photos, favorite colors and themes, and generate a list of suitable artworks based on the results."

[1043] In this way, the system aims to increase user satisfaction by suggesting the most suitable artwork based on the user's tastes and interior style.

[1044] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1045] Step 1:

[1046] User login

[1047] Input: The user enters their email address and password into the device.

[1048] Processing: The terminal sends the input data to the server, which checks it against a database and performs authentication.

[1049] Output: If authentication is successful, the server sends an authentication success message to the terminal and the user is redirected to the main screen.

[1050] What happens: The server runs an SQL query to match the user data in the database, and if there is a match, the authentication is passed.

[1051] Step 2:

[1052] Enter and submit initial registration information

[1053] Input: The user enters basic information into the device, such as their name, email address, hobbies, interior style, and budget.

[1054] Processing: The terminal sends the input data to the server, which stores the data in a database.

[1055] Output: The server sends a registration complete message to the terminal.

[1056] Specific operation: The server structures the received data and stores it in a database using SQL.

[1057] Step 3:

[1058] Generate your initial survey and collect responses

[1059] Input: The server receives the user's basic information and generates an initial survey.

[1060] Processing: The initial survey questions are sent to the device, which displays them to the user, who answers them.

[1061] Output: Sends the user's answer data to the server, which analyzes the received answer data and updates the preference data.

[1062] Specific operation: The server dynamically generates a questionnaire template based on basic information, then analyzes it using an NLP model and updates preference data.

[1063] Step 4:

[1064] Uploading and parsing media files

[1065] Input: A user uploads media files such as images, videos, and music.

[1066] Processing: The device sends the media file to the server, which uses image analysis algorithms (e.g., TensorFlow or OpenCV) to evaluate color and style. Similarly, music and video data can be analyzed to extract genres and themes.

[1067] Output: The server generates the analysis result data.

[1068] Specific operation: The server performs hue analysis on the received image data using OpenCV, and performs genre analysis on music files using the BERT model.

[1069] Step 5:

[1070] Creating and updating user preference models

[1071] Input: The server receives basic information, survey responses, and media analysis results.

[1072] Processing: Integrating this data to generate and update a model of user preferences.

[1073] Output: Save the latest preference model data to the database.

[1074] What it does: The server uses statistical models and machine learning algorithms to integrate multiple data sources and generate a model of user preferences.

[1075] Step 6:

[1076] Proposal of the best artwork

[1077] Input: The server bases its current preference model on it.

[1078] Processing: Search the database for the most suitable artwork and select multiple candidates using recommendation algorithms (e.g. collaborative filtering, content-based filtering).

[1079] Output: The server generates a list of suggested artworks for the user, along with explanatory comments for each.

[1080] Specific operation: The server uses an SQL query to search the database for artworks that match the preference model and generates comments to present them to the user.

[1081] Step 7:

[1082] View and purchase artwork

[1083] Input: User reviews a list of suggested artworks.

[1084] Action: Select the artwork you like and view more information and a link to purchase.

[1085] Output: If the customer wishes to proceed with the purchase, proceed to the purchase screen. If not, display the feedback input screen.

[1086] Specific operation: The terminal dynamically generates and displays the purchase procedure screen or feedback input screen according to the user's selection.

[1087] Step 8:

[1088] Collecting feedback and updating preference data

[1089] Input: Users enter feedback on artwork.

[1090] Processing: The device sends the feedback data to the server, which analyzes the feedback and updates the user's preference data.

[1091] Output: Save the updated preference data to the database.

[1092] Specific operation: The server analyzes the feedback data using natural language processing technology and saves the new data in the database to update the preference model again.

[1093] (Application example 1)

[1094] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1095] Conventional art recommendation systems make recommendations based solely on the user's preferences and interior style information, which tends to neglect the in-store experience. This makes it difficult to match the art pieces the user actually sees in the store, which reduces convenience when making purchasing decisions. Furthermore, since real-time recommendations based on the user's behavior and preferences in the store are not made, there is also the problem of customer satisfaction not being sufficiently improved.

[1096] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1097] In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for suitable artworks based on the preference model, means for presenting the searched artworks to the user and explaining why the artworks are suitable, and means for the user to take an image of an object in a store and suggest suitable artworks for the store. This allows for quick and accurate matching with artworks the user has actually seen in the physical store, making it possible to suggest suitable artworks that suit the user's preferences in real time.

[1098] "User preference information" refers to data about the user's preferences, such as their preferred art style, color, and theme.

[1099] "Interior style information" is information about the design and decoration style that the user prefers in their living space.

[1100] "Image, video or music analysis" refers to the process of analyzing media data provided by users to understand their content and characteristics.

[1101] "Detailed user preferences" are the result of a detailed analysis of the specific types of artwork that users prefer.

[1102] A "preference model" is a data model that represents a user's preferences and is generated based on the user's preference information and interior style information.

[1103] "Database" refers to a data storage system for storing and managing user information and artwork information.

[1104] A "searching means" is a function that extracts information from a database based on specific conditions.

[1105] An "art work" is an artistic creation such as a painting, sculpture, or photograph.

[1106] "Means for users to take pictures of objects within the store" refers to a function that allows users to take pictures of artwork within the store using a smartphone, camera, etc.

[1107] "Means of suggestion" is a function that recommends appropriate art works to users based on collected data and analysis results.

[1108] The system of this invention, "ArtShop Assistant," is a system that suggests artworks that match the user's taste in real time in a physical store. The main components of this system are a server and a terminal (smartphone) used by the user.

[1109] In the initial setup, the user logs in to the system using a terminal and registers as a user. The user's basic information, such as name, email address, hobbies, interior style, and budget, is sent from the terminal to the server and stored in a database. The server generates an initial questionnaire and displays the questions to the user in an interactive format via the terminal. The user enters their answers, and the data is sent back to the server, updating the user's preference data.

[1110] When a user searches for an artwork in a store, they use their device to take a picture of the object. This image data is sent to a server, which then uses an image analysis algorithm (e.g., OpenCV) to analyze the image's color and style. The analysis results are then reflected back into the user's preference model, updating it as detailed preference data.

[1111] The server searches the database for the most suitable artwork based on the user's preference model and the evaluation results from image analysis, and evaluates the search results using a generative AI model. This generative AI model recommends appropriate artworks in real time based on the user's prompt. Typical examples of prompts are as follows:

[1112] User: I like modern interiors with darker tones. I'm looking for abstract art with a predominant blue color.

[1113] AI prompt: Suggest artwork that takes into account the user's interior style (dark tones, modern) and preferences (blue, abstract art).

[1114] The device displays a list of artworks sent from the server and the reasons why each artwork is the best fit. The user can select their favorite artwork from the list and proceed to the purchase process. Even if they do not purchase, they can enter feedback, which is sent to the server and used to further update the user's preference data.

[1115] This allows users to find artworks that match their tastes in real time in a physical store. The main hardware used is a smartphone and a server, and the software includes a web application framework using Flask on the server side, image analysis libraries such as OpenCV, and generative AI models. This improves the user's in-store experience and makes it easier to recommend and purchase the most suitable artworks.

[1116] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1117] Step 1:

[1118] When a user logs in for the first time, the device displays a user registration form. The user enters basic information such as name, email address, hobbies, interior style, and budget, and the device sends this information to the server, which stores the received data in a database.

[1119] Input: User's basic information (name, email address, hobbies, interior style, budget)

[1120] Output: User information stored in the database

[1121] Step 2:

[1122] The server generates an initial questionnaire based on the user information and displays the questions interactively to the user through the terminal. The user answers the questions, and the terminal sends the answers to the server. The server analyzes the answers and updates the user's preference data.

[1123] Input: User's survey response

[1124] Output: Updated user preference data

[1125] Step 3:

[1126] When a user searches for an artwork in a store, they take a picture of the object with their device. This image data is sent to a server, which then uses image analysis algorithms (such as OpenCV) to analyze the image's color and style. The analysis results are reflected in the user's preference data.

[1127] Input: A photographed image of the object

[1128] Output: Analyzed image color and style data

[1129] Step 4:

[1130] The server searches the database for the most suitable artwork based on the user's preference model and the results of image analysis, and then uses a generative AI model to recommend suitable artworks in real time, matching them with the user's prompts.

[1131] Input: User preference model, image analysis results

[1132] Output: A list of the best artworks

[1133] Step 5:

[1134] The server generates a list of artworks and the reasons why they are the best, and sends it to the device. The device displays this information to the user, who can then select the artwork they like from the list.

[1135] Input: A list of the best artworks, and why they're the best.

[1136] Output: A list of artworks displayed to the user

[1137] Step 6:

[1138] When the user selects a piece of art they like, the device sends the selection to the server, which then generates a checkout link and sends it to the device. The user can click the link to complete the purchase. Even if the user does not purchase, they can still enter feedback, which is sent from the device to the server, which then updates the preference data.

[1139] Input: User's artwork selection or feedback

[1140] Output: Checkout page link, updated preference data

[1141] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1142] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is mainly composed of a server, a terminal, and an emotion engine, and users access the system through their terminal.

[1143] When the system starts, the user logs in to the system using a terminal. On first use, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal sends this entered data to the server, which then stores the received data in a database.

[1144] After collecting user information, the server generates initial survey questions for the user based on this information. The device displays the questions interactively to the user, who answers them. The answers are sent from the device to the server, which analyzes them and updates the user's preference data. The device then prompts the user to input images, videos, music, and emotional states through an emotion engine. After the user uploads these media and provides emotional information, the device sends the data to the server.

[1145] The server runs uploaded images through image analysis algorithms to evaluate color and style. Similarly, it analyzes music and video data to extract genres and themes. The emotion engine analyzes the user's current emotional state based on facial expressions and voice, and sends that information to the server.

[1146] The server integrates these analysis results to generate and update a detailed user preference model. Based on the preference model, the server searches the database for the most suitable artworks. From the search results, the server selects multiple candidates and generates comments explaining why each artwork is the best fit for the user.

[1147] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[1148] As a concrete example, the following scenario can be considered.

[1149] Example: Art suggestions based on tastes and emotional states

[1150] 1. User: Access the system and select "Art to watch when you want to relax."

[1151] 2. On the device, the user is prompted to upload photos and music to relax with.

[1152] 3. User: Upload photos of the music you like to listen to when relaxing or the interior design.

[1153] 4. Server: Analyzes the uploaded data and evaluates the color and mood of the music.

[1154] 5. Emotion engine: Analyzes the user's current emotional state and sends it to the server as a "want to relax" state.

[1155] 6. Server: Using the collected information, search the database for suitable artworks for relaxation.

[1156] 7. Server: Select multiple candidates from the search results and write a description of each artwork explaining why it is suitable for the user.

[1157] 8. Terminal: Presents the user with a list of candidate artworks and descriptions.

[1158] 9. User: Selects the artwork they like and proceeds to checkout.

[1159] In this way, the system takes into account the user's tastes, interior style, and emotional state to suggest the most suitable artwork, deepening their understanding of art and encouraging them to make purchases.

[1160] The processing flow will be explained below.

[1161] Step 1:

[1162] A user logs into the system using a terminal.

[1163] Step 2:

[1164] The device displays a user registration form when used for the first time.

[1165] Step 3:

[1166] The user enters basic information such as name, email address, hobbies, interior style, and budget.

[1167] Step 4:

[1168] The terminal transmits the input data to the server.

[1169] Step 5:

[1170] The server stores the received data in a database.

[1171] Step 6:

[1172] The server generates questions for the user based on the results of the initial survey.

[1173] Step 7:

[1174] The device interactively asks the user questions (e.g., "What is your favorite genre of art?").

[1175] Step 8:

[1176] The user enters the answer.

[1177] Step 9:

[1178] The device sends the response to the server.

[1179] Step 10:

[1180] The server analyzes the responses and updates the user's preference data.

[1181] Step 11:

[1182] The device prompts the user to input images, videos, music, and emotional states using an emotion engine.

[1183] Step 12:

[1184] Users upload photos, videos, music and emotional information.

[1185] Step 13:

[1186] The terminal transmits the uploaded data to the server.

[1187] Step 14:

[1188] The server runs the uploaded photos through image analysis algorithms to evaluate color and style.

[1189] Step 15:

[1190] The server analyzes the music and video data and extracts genres and themes.

[1191] Step 16:

[1192] The emotion engine analyzes the user's emotional state from their facial expressions and voice and sends the data to the server.

[1193] Step 17:

[1194] The server integrates the analysis results (images, music, videos, emotions) to generate and update a detailed preference model of the user.

[1195] Step 18:

[1196] The server searches the database for the most suitable artwork based on the preference model.

[1197] Step 19:

[1198] The server generates a list of artworks as search results.

[1199] Step 20:

[1200] The server generates comments explaining why each artwork is suitable for the user.

[1201] Step 21:

[1202] The terminal displays the generated list of artworks and explanatory comments to the user.

[1203] Step 22:

[1204] The user reviews the proposed artworks and selects the one they like best.

[1205] Step 23:

[1206] Your device will display detailed information and a link to purchase the selected title.

[1207] Step 24:

[1208] The user completes a purchase or provides feedback about the work.

[1209] Step 25:

[1210] The device sends feedback or purchase information to the server.

[1211] Step 26:

[1212] The server receives the feedback and updates the user's preference data.

[1213] This series of steps suggests the most suitable artworks based on the user's tastes, interior style, and even emotional state, helping them better understand art and encourage them to make purchases.

[1214] Example 2

[1215] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1216] Conventional art suggestion systems have the problem of not being able to suggest the most suitable artworks because they do not fully consider the user's preferences or emotional state. In addition, they lack a system to reflect changes in user preferences, making it difficult to respond to individual needs.

[1217] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through a questionnaire, means for analyzing image, music, or video media data, means for analyzing the user's emotional state, means for integrating the analysis results to generate and update a user preference model, means for searching a database for an optimal visual artwork based on the preference model, and means for presenting the searched visual artwork to the user and explaining why the artwork is optimal. This makes it possible to suggest optimal artworks that take the user's preferences and emotional state into consideration.

[1218] "User preference information and interior style information" is information related to the user's hobbies and preferences, and the interior design style of the user's home, office, etc.

[1219] A "survey" is a set of questions distributed to users to understand their detailed hobbies and preferences.

[1220] "Image, music or video media data" refers to image files, music files and video files uploaded by users for relaxation or related to their hobbies.

[1221] "User's emotional state" is information that indicates the user's current emotional or psychological state.

[1222] "Means for integrating analysis results to generate and update a user preference model" refers to the process of analyzing collected data to create and update an individual preference model that reflects the user's tastes and emotional state.

[1223] The "means for retrieving optimal visual artworks from a database based on a preference model" is a process for retrieving optimal artworks from a database using a user's preference model.

[1224] "A means of presenting visual artworks resulting from a search result to a user and explaining why each artwork is the most appropriate" is a process of presenting artworks retrieved from a database to a user and explaining why each artwork is the most appropriate for the user.

[1225] "Feedback Information" refers to the opinions and ratings you provide regarding the artworks you select or purchase.

[1226] The present invention is a system that performs detailed analysis based on a user's preference information and interior style information to suggest the most suitable artwork. The system mainly includes a server, a terminal, and an emotion engine. The user accesses the system through the terminal.

[1227] System configuration

[1228] 1. Means of collecting user preference information and interior style information

[1229] Device: When using for the first time, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget.

[1230] Terminal: Sends the entered information to the server.

[1231] Server: Receives user information and stores it in a database.

[1232] 2. A means of analyzing detailed user preferences through surveys

[1233] Server: Generates initial survey questions based on user information.

[1234] Terminal: Questions are presented to the user interactively, and the user answers them.

[1235] Terminal: Sends survey responses to the server.

[1236] Server: Analyzes the received survey responses and updates the user preference data.

[1237] 3. Means for analyzing image, music or video media data

[1238] Terminal: Provides an interface for users to upload photos of their favorite music and interior design when relaxing.

[1239] User: Upload media data such as images, music, and videos.

[1240] Terminal: Sends the uploaded data to the server.

[1241] Server: Analyzes the received image data using an image analysis algorithm (e.g., OpenCV).

[1242] Server: Analyzes music and video data using a library (e.g., LIBROSA) to extract genres and themes.

[1243] 4. A means of analyzing the user's emotional state

[1244] Emotion engine: Analyzes the user's current emotional state from their facial expressions and voice, and sends that information to the server.

[1245] 5. Means of integrating the analysis results to generate and update user preference models

[1246] Server: Integrates the analysis results of media data and emotional information to generate and update a detailed user preference model.

[1247] 6. A method for searching a database for suitable visual artworks based on a preference model

[1248] Server: Search for visual artworks from a database using the preference model (e.g., using Elasticsearch).

[1249] 7. A way to present users with visual art results and explain why they are the best.

[1250] Server: Selects candidate works from the search results and uses a generative AI model to generate comments explaining why each one is the best.

[1251] Terminal: Displays a list of artworks and explanatory comments to the user.

[1252] Specific examples

[1253] Example: Art suggestions based on tastes and emotional states

[1254] 1. User: Access the system and select "Art to watch when you want to relax."

[1255] 2. On the device, the user is prompted to upload photos and music to relax with.

[1256] 3. User: Upload photos of the music you like to listen to when relaxing or the interior design.

[1257] 4. Server: Analyzes the uploaded data and evaluates the color and musical atmosphere (using OpenCV for image data and LIBROSA for music data).

[1258] 5. Emotion engine: Analyzes the user's current emotional state and sends it to the server as a "want to relax" state.

[1259] 6. Server: Based on the collected information, search the database for suitable artworks for relaxation (using Elasticsearch).

[1260] 7. Server: Selects multiple candidates from the search results and uses a generative AI model to compile a description of why each artwork is suitable for the user.

[1261] 8. Terminal: Presents the user with a list of candidate artworks and descriptions.

[1262] 9. User: Selects the artwork they like and proceeds to checkout.

[1263] Prompt Sentence Examples

[1264] plain text

[1265] Describe art suggestions based on user preferences, interior style, and emotional state. Consider the following:

[1266] User basic information (name, email address, hobbies, interior style, budget)

[1267] User-uploaded media data (images, music, videos)

[1268] Emotion engine analysis results

[1269] Pick the best piece of art and explain why it's the best fit for your users.

[1270] In this way, the system of the present invention can effectively suggest the most suitable artworks by reflecting the preferences and emotional state of each user in detail.

[1271] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1272] Step 1:

[1273] A user accesses the system and logs in by entering their name and password in the login form. When using the system for the first time, the terminal displays a user registration form, and the user enters basic information such as their name, email address, hobbies, interior style, and budget. The entered information is sent from the terminal to the server.

[1274] Input: Basic information such as name, email address, hobbies, interior style, budget, etc.

[1275] Output: Basic information about the user sent to the server

[1276] Step 2:

[1277] The server receives the user's basic information sent from the device and stores it in a database, which is used for subsequent analysis.

[1278] Input: Basic user information sent from the device

[1279] Output: Basic user information stored in the database

[1280] Step 3:

[1281] The server generates initial survey questions based on the user information, and sends the generated questions to the terminal, which then displays the survey questions to the user in an interactive format. The user answers the questions.

[1282] Input: Basic user information stored in the database

[1283] Output: Generated initial survey questions

[1284] Step 4:

[1285] The user answers the questionnaire and enters the answers into the terminal, which then sends the user's answers to the server.

[1286] Input: User's survey response

[1287] Output: Survey responses sent to the server

[1288] Step 5:

[1289] The server analyzes the received survey responses and updates the user's preference data using a text analysis algorithm.

[1290] Input: User's survey response

[1291] Output: Updated user preference data

[1292] Step 6:

[1293] The device provides a user with an interface that prompts the user to upload media data such as images, videos, and music, and input emotional information. The user uploads relevant media data and inputs emotional information, and these data are transmitted from the device to the server.

[1294] Input: User-uploaded media data and emotional information

[1295] Output: Media data and emotion information sent to the server

[1296] Step 7:

[1297] The server analyzes the received image data using an image analysis algorithm (e.g., OpenCV). Music and video data is analyzed using an analysis library (e.g., LIBROSA) to extract genres and themes. The emotion engine also analyzes the current emotional state from facial expressions and voice, and this information is sent to the server.

[1298] Input: image data, music data, video data, emotional information

[1299] Output: Analysis results (color, style, genre, theme, emotional state)

[1300] Step 8:

[1301] The server integrates the analysis results and generates and updates a detailed user preference model using data fusion techniques.

[1302] Input: Analysis results

[1303] Output: User preference model

[1304] Step 9:

[1305] The server searches the database for the most suitable visual artworks based on the preference model, using a database search engine (e.g., Elasticsearch).

[1306] Input: User preference model

[1307] Output: Searched visual artwork candidates

[1308] Step 10:

[1309] The server selects candidate artworks from the search results and uses a generative AI model to generate comments explaining why each is the best fit. The generated comments and list of artworks are then sent to the device.

[1310] Input: Search results for visual artworks

[1311] Output: Generated explanatory comments and artwork list

[1312] Step 11:

[1313] The device displays the generated explanatory comments and a list of artworks to the user.

[1314] Input: Generated descriptive comments and artwork list

[1315] Output: A list of artworks and explanatory comments displayed to the user

[1316] Step 12:

[1317] The user reviews the proposed artworks and selects the one they like. Detailed information and a link to purchase the selected artwork are displayed. If the user does not purchase, they can enter their opinion through a feedback form. The feedback or purchase information is sent from the device to the server.

[1318] Input: User's work selection, feedback or purchase information

[1319] Output: Feedback or purchase information sent to the server

[1320] Step 13:

[1321] The server receives the feedback or purchase information and updates the user's preference data again.

[1322] Input: Feedback or Purchase Information

[1323] Output: Updated user preference data

[1324] (Application example 2)

[1325] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1326] In conventional logistics centers, real-time analysis of worker efficiency and emotional state is not adequately performed, and optimal work instructions are not provided according to the state of each worker, resulting in problems such as reduced work efficiency and mistakes due to overwork.In addition, dynamic task adjustment based on employee feedback is not performed, making it difficult to optimize the entire operation.

[1327] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for suitable artworks based on the preference model, means for presenting the searched artworks to the user and explaining why the artworks are suitable, and means for analyzing the user's work efficiency and emotional state using a smart device and providing optimal task instructions. This makes it possible to dynamically adjust work instructions in real time and provide optimal instructions according to the status of each worker.

[1328] "User preference information" is information that reflects the user's hobbies, preferences, and interests.

[1329] "Interior style information" is information about the interior decoration of a user's home or office.

[1330] "Image, video or music analysis" refers to the analysis of these media data to assess a user's preferences and emotional state.

[1331] "Detailed user preferences" refers to information about a user's specific hobbies and preferences, extracted from the analyzed data.

[1332] A "user preference model" is a data model that reflects a user's preferences and is generated based on the user's collected preference information.

[1333] The "optimal artwork" is the artwork selected based on the user's preference model and determined to be most suitable for the user.

[1334] A "smart device" is an advanced device that can connect to the Internet and is equipped with various sensors, and examples include smart glasses.

[1335] "Work efficiency" refers to the efficiency and productivity of workers when performing their work.

[1336] "Emotional state" refers to the user's mental state or emotions, including stress, fatigue, joy, etc.

[1337] "Optimal task instructions" are instructions that are most suitable for each task, based on the user's work efficiency and emotional state.

[1338] The system of the present invention analyzes user preference information and emotional states to provide optimal task instructions in order to improve work efficiency and the emotional state of workers in logistics centers. The system mainly consists of a server, terminals, and smart devices (e.g., smart glasses).

[1339] First, the worker (user) wears a smart device and accesses the system through that device. At this time, the user's preference information and interior style information are collected and sent to the server. The server stores this information in a database.

[1340] The server then analyzes media data such as images, videos, and music to assess the user's detailed preferences. This analysis is performed using software libraries such as OpenCV and NLTK. To analyze the user's emotional state, sensors on the smart device are used to analyze facial expressions and voice. OpenCV is used for facial expression analysis, and NLTK, a natural language processing technology, is used for voice analysis.

[1341] Based on the analysis of the user's detailed preference data and emotional state, the server generates and updates a user preference model. Based on the preference model, the server searches for the most suitable artwork from the database and presents it to the user. At the same time, the server analyzes the user's work efficiency and emotional state in real time via the smart device and provides appropriate task instructions.

[1342] As a concrete example, consider the case where an employee at a logistics center uses smart glasses. If the smart glasses detect high stress levels in the employee through facial expression analysis while they are working, the server sends an instruction message to the smart glasses saying, "Perform light work." After the work is completed, feedback from the employee is collected and stored in a database. This feedback is reflected in the next task instructions.

[1343] An example of a prompt sentence is as follows:

[1344] "Build a system that analyzes the emotional state of logistics center employees and adjusts their workload via smart glasses. Specifically, this would include a function that uses facial expression recognition and emotion analysis to determine stress and fatigue, and provides appropriate work instructions in real time."

[1345] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1346] Step 1:

[1347] The user puts on a smart device (smart glasses) and logs into the system. After logging in, the first time they use the system, they are prompted to enter their preferences (hobbies, interior style, budget, etc.). The information entered by the user is sent from the device to the server and stored in a database.

[1348] Input: User preferences (hobbies, interior style, budget, etc.)

[1349] Output: User preferences stored in a database

[1350] Step 2:

[1351] The server generates an initial questionnaire based on the user's preference information and interior style information, and sends the questions to the terminal. The terminal displays the questions to the user in an interactive format and receives the answers.

[1352] Input: User preference information and interior style information

[1353] Output: User's survey responses

[1354] Step 3:

[1355] The user's response data is sent from the device to a server and analyzed to build detailed user preference data. The user also allows the camera for facial expression analysis and microphone for voice analysis to collect emotional state data.

[1356] Input: User survey responses and emotional data

[1357] Output: Detailed preference and sentiment data

[1358] Step 4:

[1359] The server uses OpenCV and NLTK libraries for facial expression and speech analysis, which analyzes camera images and audio samples to assess the user's emotional state (e.g., stress level).

[1360] Input: Camera images and audio samples

[1361] Output: Parsed emotion data (e.g., stress level)

[1362] Step 5:

[1363] Based on the analysis results, the server generates and updates the user's preference model and stores it in the database.Then, based on the preference model, the server searches the database for the most suitable artworks and recommended tasks based on the questionnaire.

[1364] Input: Parsed emotion data and detailed preference data

[1365] Output: Updated preference model and best artwork or recommended work

[1366] Step 6:

[1367] The server then provides the user with real-time task instructions through the smart glasses, tailored to their emotional state. For example, if high stress is detected, instructions for lighter tasks will be displayed.

[1368] Input: Emotional state and preference model

[1369] Output: Task instructions displayed on smart glasses

[1370] Step 7:

[1371] After completing the task, the device collects feedback from the user and sends it to the server, which uses this information to further update the preference model and reflect it in the next task instructions.

[1372] Input: User feedback

[1373] Output: Updated preference model and next task instructions

[1374] In this way, the system of the present invention can provide optimal work instructions in real time based on the user's preference information and emotional state.

[1375] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1376] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1377] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1378] [Fourth embodiment]

[1379] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1380] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1381] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1382] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1383] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1384] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1385] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1386] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1387] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1388] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1389] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1390] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1391] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1392] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is mainly composed of a server and a terminal, and users access the system through the terminal.

[1393] When the system starts, the user logs in to the system using a terminal. On first use, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal sends this entered data to the server, which then stores the received data in a database.

[1394] After collecting user information, the server generates initial survey questions for the user based on this information. The device interactively displays the questions to the user, who answers them. The device sends the answers to the server, which analyzes them and updates the user's preference data. The device then prompts the user to upload images, videos, and music. When the user uploads these media, the device sends the data to the server.

[1395] The server runs uploaded images through image analysis algorithms to evaluate color and style. Similarly, it analyzes music and video data to extract genres and themes. It combines these analysis results to generate and update a detailed user preference model. The server then searches its database for the most suitable artwork based on the preference model. It selects multiple candidates from the search results and generates a comment explaining why each artwork is the best fit for the user.

[1396] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[1397] As a concrete example, the following scenario can be considered.

[1398] Example: Art suggestions based on hobbies and interior styles

[1399] 1. User: Enters the system and types in "art for my living room."

[1400] 2. Terminal: The user is prompted to "upload a photo of your living room."

[1401] 3. User: Uploads a photo of their living room.

[1402] 4. Server: Analyzes the uploaded photos and evaluates their color and style (modern, classic, etc.).

[1403] 5. Device: Display a question to the user: "What are your favorite colors or elements (abstract paintings, small objects, specific themes, etc.)?"

[1404] 6. User: "I like the color blue and I like abstract art."

[1405] 7. Server: Based on the collected information, search the database for abstract paintings with blue as the main color.

[1406] 8. Server: Select multiple candidates from the search results and write a description of why each piece of art would be suitable for the user's living room.

[1407] 9. Terminal: Presents the user with a list of candidate artworks and descriptions.

[1408] 10. User: Selects the artwork they like and proceeds to checkout.

[1409] In this way, the system suggests the most suitable artworks based on the user's tastes and interior style, encouraging them to purchase art.

[1410] The processing flow will be explained below.

[1411] Step 1:

[1412] A user logs into the system using a terminal.

[1413] Step 2:

[1414] The device displays a user registration form when used for the first time.

[1415] Step 3:

[1416] The user enters basic information such as name, email address, hobbies, interior style, and budget.

[1417] Step 4:

[1418] The terminal transmits the input data to the server.

[1419] Step 5:

[1420] The server stores the received data in a database.

[1421] Step 6:

[1422] The server generates questions for the user based on the results of the initial survey.

[1423] Step 7:

[1424] The device interactively asks the user questions (e.g., "What is your favorite genre of art?").

[1425] Step 8:

[1426] The user enters the answer.

[1427] Step 9:

[1428] The device sends the response to the server.

[1429] Step 10:

[1430] The server analyzes the responses and updates the user's preference data.

[1431] Step 11:

[1432] The device prompts the user to upload images, videos, and music.

[1433] Step 12:

[1434] Users upload photos, videos, and music.

[1435] Step 13:

[1436] The terminal transmits the uploaded data to the server.

[1437] Step 14:

[1438] The server runs the uploaded photos through image analysis algorithms to evaluate color and style.

[1439] Step 15:

[1440] The server analyzes the music and video data and extracts genres and themes.

[1441] Step 16:

[1442] The server integrates the analysis results to generate and update a detailed preference model of the user.

[1443] Step 17:

[1444] The server searches the database for the most suitable artwork based on the preference model.

[1445] Step 18:

[1446] The server generates a list of artworks as search results.

[1447] Step 19:

[1448] The server generates comments explaining why each artwork is suitable for the user.

[1449] Step 20:

[1450] The terminal displays the generated list of artworks and explanatory comments to the user.

[1451] Step 21:

[1452] The user reviews the proposed artworks and selects the one they like best.

[1453] Step 22:

[1454] Your device will display detailed information and a link to purchase the selected title.

[1455] Step 23:

[1456] The user completes a purchase or provides feedback about the work.

[1457] Step 24:

[1458] The device sends feedback or purchase information to the server.

[1459] Step 25:

[1460] The server receives the feedback and updates the user's preference data.

[1461] This series of steps suggests artworks that best suit the user's tastes and interior style, deepening their understanding of art and encouraging them to make a purchase.

[1462] Example 1

[1463] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1464] Today's consumers have a wide variety of tastes and interior styles, making it difficult to select artworks that suit them. Finding artworks that match a user's tastes and interior styles takes a lot of time and effort. Furthermore, because user preferences change over time, it is necessary to constantly make suggestions that reflect the latest preferences. Given this situation, there is a need for a method to analyze detailed user preference data and efficiently suggest the most suitable artworks.

[1465] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1466] In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for the most suitable artwork based on the preference model, means for presenting the searched artwork to the user and explaining why the artwork is the most suitable, and means for collecting user feedback or purchase information and updating the preference model, thereby making it possible to suggest the most suitable artwork based on the user's tastes and interior style.

[1467] "Preference information" is information about a user's personal preferences, such as their preferred colors, styles, genres, and themes.

[1468] "Interior style information" refers to information about the decoration and design of a user's home or living space, such as modern, classic, minimalist, and other styles.

[1469] "Analysis means" refers to algorithms and software for analyzing data such as images, videos, and music, and is a means for analyzing user preferences in detail.

[1470] A "preference model" is a data model constructed based on collected and analyzed user preference information and interior style information, and reflects the user's preferences.

[1471] A "database" is a database management system for storing information about users and artworks, and is a means of searching and storing information.

[1472] "Search tools" refers to algorithms or software that search the database for suitable artworks based on preference models.

[1473] "Presentation means" refers to the means by which the artworks and descriptions of the search results are displayed to the user, typically via a user interface.

[1474] "Feedback Information" refers to the opinions and ratings that users provide about selected artworks and suggestions, which the system uses to update its preference model.

[1475] "Purchase Information" means information relating to the actual purchase of a selected artwork by a User, including transaction details and history.

[1476] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is primarily composed of a server and a terminal, and users access the system through the terminal. Specific embodiments for implementing the present invention are described below.

[1477] First, the user logs in to the system using a terminal. When using for the first time, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal then sends this entered data to the server. The server then stores the received data in a database. The database uses a general database management system (DBMS), and can be, for example, a database using SQL.

[1478] After collecting user information, the server generates initial survey questions for the user based on this information. The initial survey is intended to collect more detailed user preference data. The device displays the questions to the user in an interactive format, and the user answers them. The answers are sent from the device to the server, which analyzes them and updates the user preference data. The server's analysis function can use Python-based AI models (such as TensorFlow and OpenCV).

[1479] The device then prompts the user to upload images, videos, and music. Once the user uploads these media files, the device sends the data to a server. The server runs the uploaded images through image analysis algorithms (TensorFlow and OpenCV) to evaluate color and style. Similarly, the server analyzes music and video data to extract genres and themes. This analysis uses natural language processing techniques (e.g., SpaCy and the BERT model).

[1480] The server combines these analysis results to generate and update a detailed user preference model. Based on the preference model, the server searches the database for the most suitable artworks. The algorithm used for the search can use recommendation techniques (e.g., collaborative filtering or content-based filtering). From the search results, multiple candidates are selected and a comment is generated explaining why each artwork is the best fit for the user.

[1481] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[1482] As a concrete example, below we show the steps for proposing art based on hobbies and interior style.

[1483] 1. User: Enters the system and types in "art for my living room."

[1484] 2. Terminal: The user is prompted to "upload a photo of your living room."

[1485] 3. User: Uploads a photo of their living room.

[1486] 4. Server: Analyzes the uploaded photos and evaluates their color and style (modern, classic, etc.).

[1487] 5. Device: Display a question to the user: "What are your favorite colors or elements (abstract paintings, small objects, specific themes, etc.)?"

[1488] 6. User: "I like the color blue and I like abstract art."

[1489] 7. Server: Based on the collected information, search the database for abstract paintings with blue as the main color.

[1490] 8. Server: Select multiple candidates from the search results and write a description of why each piece of art would be suitable for the user's living room.

[1491] 9. Terminal: Presents the user with a list of candidate artworks and descriptions.

[1492] 10. User: Select the artwork they like and proceed to checkout.

[1493] An example of a prompt sentence to input to the generative AI model is as follows:

[1494] "Collect information about the user's tastes and interior style, and suggest suitable artworks. For example, analyze the user's interior photos, favorite colors and themes, and generate a list of suitable artworks based on the results."

[1495] In this way, the system aims to increase user satisfaction by suggesting the most suitable artwork based on the user's tastes and interior style.

[1496] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1497] Step 1:

[1498] User login

[1499] Input: The user enters their email address and password into the device.

[1500] Processing: The terminal sends the input data to the server, which checks it against a database and performs authentication.

[1501] Output: If authentication is successful, the server sends an authentication success message to the terminal and the user is redirected to the main screen.

[1502] What happens: The server runs an SQL query to match the user data in the database, and if there is a match, the authentication is passed.

[1503] Step 2:

[1504] Enter and submit initial registration information

[1505] Input: The user enters basic information into the device, such as their name, email address, hobbies, interior style, and budget.

[1506] Processing: The terminal sends the input data to the server, which stores the data in a database.

[1507] Output: The server sends a registration complete message to the terminal.

[1508] Specific operation: The server structures the received data and stores it in a database using SQL.

[1509] Step 3:

[1510] Generate your initial survey and collect responses

[1511] Input: The server receives the user's basic information and generates an initial survey.

[1512] Processing: The initial survey questions are sent to the device, which displays them to the user, who answers them.

[1513] Output: Sends the user's answer data to the server, which analyzes the received answer data and updates the preference data.

[1514] Specific operation: The server dynamically generates a questionnaire template based on basic information, then analyzes it using an NLP model and updates preference data.

[1515] Step 4:

[1516] Uploading and parsing media files

[1517] Input: A user uploads media files such as images, videos, and music.

[1518] Processing: The device sends the media file to the server, which uses image analysis algorithms (e.g., TensorFlow or OpenCV) to evaluate color and style. Similarly, music and video data can be analyzed to extract genres and themes.

[1519] Output: The server generates the analysis result data.

[1520] Specific operation: The server performs hue analysis on the received image data using OpenCV, and performs genre analysis on music files using the BERT model.

[1521] Step 5:

[1522] Creating and updating user preference models

[1523] Input: The server receives basic information, survey responses, and media analysis results.

[1524] Processing: Integrating this data to generate and update a model of user preferences.

[1525] Output: Save the latest preference model data to the database.

[1526] What it does: The server uses statistical models and machine learning algorithms to integrate multiple data sources and generate a model of user preferences.

[1527] Step 6:

[1528] Proposal of the best artwork

[1529] Input: The server bases its current preference model on it.

[1530] Processing: Search the database for the most suitable artwork and select multiple candidates using recommendation algorithms (e.g. collaborative filtering, content-based filtering).

[1531] Output: The server generates a list of suggested artworks for the user, along with explanatory comments for each.

[1532] Specific operation: The server uses an SQL query to search the database for artworks that match the preference model and generates comments to present them to the user.

[1533] Step 7:

[1534] View and purchase artwork

[1535] Input: User reviews a list of suggested artworks.

[1536] Action: Select the artwork you like and view more information and a link to purchase.

[1537] Output: If the customer wishes to proceed with the purchase, proceed to the purchase screen. If not, display the feedback input screen.

[1538] Specific operation: The terminal dynamically generates and displays the purchase procedure screen or feedback input screen according to the user's selection.

[1539] Step 8:

[1540] Collecting feedback and updating preference data

[1541] Input: Users enter feedback on artwork.

[1542] Processing: The device sends the feedback data to the server, which analyzes the feedback and updates the user's preference data.

[1543] Output: Save the updated preference data to the database.

[1544] Specific operation: The server analyzes the feedback data using natural language processing technology and saves the new data in the database to update the preference model again.

[1545] (Application example 1)

[1546] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1547] Conventional art recommendation systems make recommendations based solely on the user's preferences and interior style information, which tends to neglect the in-store experience. This makes it difficult to match the art pieces the user actually sees in the store, which reduces convenience when making purchasing decisions. Furthermore, since real-time recommendations based on the user's behavior and preferences in the store are not made, there is also the problem of customer satisfaction not being sufficiently improved.

[1548] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1549] In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for suitable artworks based on the preference model, means for presenting the searched artworks to the user and explaining why the artworks are suitable, and means for the user to take an image of an object in a store and suggest suitable artworks for the store. This allows for quick and accurate matching with artworks the user has actually seen in the physical store, making it possible to suggest suitable artworks that suit the user's preferences in real time.

[1550] "User preference information" refers to data about the user's preferences, such as their preferred art style, color, and theme.

[1551] "Interior style information" is information about the design and decoration style that the user prefers in their living space.

[1552] "Image, video or music analysis" refers to the process of analyzing media data provided by users to understand their content and characteristics.

[1553] "Detailed user preferences" are the result of a detailed analysis of the specific types of artwork that users prefer.

[1554] A "preference model" is a data model that represents a user's preferences and is generated based on the user's preference information and interior style information.

[1555] "Database" refers to a data storage system for storing and managing user information and artwork information.

[1556] A "searching means" is a function that extracts information from a database based on specific conditions.

[1557] An "art work" is an artistic creation such as a painting, sculpture, or photograph.

[1558] "Means for users to take pictures of objects within the store" refers to a function that allows users to take pictures of artwork within the store using a smartphone, camera, etc.

[1559] "Means of suggestion" is a function that recommends appropriate art works to users based on collected data and analysis results.

[1560] The system of this invention, "ArtShop Assistant," is a system that suggests artworks that match the user's taste in real time in a physical store. The main components of this system are a server and a terminal (smartphone) used by the user.

[1561] In the initial setup, the user logs in to the system using a terminal and registers as a user. The user's basic information, such as name, email address, hobbies, interior style, and budget, is sent from the terminal to the server and stored in a database. The server generates an initial questionnaire and displays the questions to the user in an interactive format via the terminal. The user enters their answers, and the data is sent back to the server, updating the user's preference data.

[1562] When a user searches for an artwork in a store, they use their device to take a picture of the object. This image data is sent to a server, which then uses an image analysis algorithm (e.g., OpenCV) to analyze the image's color and style. The analysis results are then reflected back into the user's preference model, updating it as detailed preference data.

[1563] The server searches the database for the most suitable artwork based on the user's preference model and the evaluation results from image analysis, and evaluates the search results using a generative AI model. This generative AI model recommends appropriate artworks in real time based on the user's prompt. Typical examples of prompts are as follows:

[1564] User: I like modern interiors with darker tones. I'm looking for abstract art with a predominant blue color.

[1565] AI prompt: Suggest artwork that takes into account the user's interior style (dark tones, modern) and preferences (blue, abstract art).

[1566] The device displays a list of artworks sent from the server and the reasons why each artwork is the best fit. The user can select their favorite artwork from the list and proceed to the purchase process. Even if they do not purchase, they can enter feedback, which is sent to the server and used to further update the user's preference data.

[1567] This allows users to find artworks that match their tastes in real time in a physical store. The main hardware used is a smartphone and a server, and the software includes a web application framework using Flask on the server side, image analysis libraries such as OpenCV, and generative AI models. This improves the user's in-store experience and makes it easier to recommend and purchase the most suitable artworks.

[1568] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1569] Step 1:

[1570] When a user logs in for the first time, the device displays a user registration form. The user enters basic information such as name, email address, hobbies, interior style, and budget, and the device sends this information to the server, which stores the received data in a database.

[1571] Input: User's basic information (name, email address, hobbies, interior style, budget)

[1572] Output: User information stored in the database

[1573] Step 2:

[1574] The server generates an initial questionnaire based on the user information and displays the questions interactively to the user through the terminal. The user answers the questions, and the terminal sends the answers to the server. The server analyzes the answers and updates the user's preference data.

[1575] Input: User's survey response

[1576] Output: Updated user preference data

[1577] Step 3:

[1578] When a user searches for an artwork in a store, they take a picture of the object with their device. This image data is sent to a server, which then uses image analysis algorithms (such as OpenCV) to analyze the image's color and style. The analysis results are reflected in the user's preference data.

[1579] Input: A photographed image of the object

[1580] Output: Analyzed image color and style data

[1581] Step 4:

[1582] The server searches the database for the most suitable artwork based on the user's preference model and the results of image analysis, and then uses a generative AI model to recommend suitable artworks in real time, matching them with the user's prompts.

[1583] Input: User preference model, image analysis results

[1584] Output: A list of the best artworks

[1585] Step 5:

[1586] The server generates a list of artworks and the reasons why they are the best, and sends it to the device. The device displays this information to the user, who can then select the artwork they like from the list.

[1587] Input: A list of the best artworks, and why they're the best.

[1588] Output: A list of artworks displayed to the user

[1589] Step 6:

[1590] When the user selects a piece of art they like, the device sends the selection to the server, which then generates a checkout link and sends it to the device. The user can click the link to complete the purchase. Even if the user does not purchase, they can still enter feedback, which is sent from the device to the server, which then updates the preference data.

[1591] Input: User's artwork selection or feedback

[1592] Output: Checkout page link, updated preference data

[1593] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1594] The system of the present invention collects user preference information and interior style information, performs detailed analysis, and then proposes the most suitable artwork. The system is mainly composed of a server, a terminal, and an emotion engine, and users access the system through their terminal.

[1595] When the system starts, the user logs in to the system using a terminal. On first use, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget. The terminal sends this entered data to the server, which then stores the received data in a database.

[1596] After collecting user information, the server generates initial survey questions for the user based on this information. The device displays the questions interactively to the user, who answers them. The answers are sent from the device to the server, which analyzes them and updates the user's preference data. The device then prompts the user to input images, videos, music, and emotional states through an emotion engine. After the user uploads these media and provides emotional information, the device sends the data to the server.

[1597] The server runs uploaded images through image analysis algorithms to evaluate color and style. Similarly, it analyzes music and video data to extract genres and themes. The emotion engine analyzes the user's current emotional state based on facial expressions and voice, and sends that information to the server.

[1598] The server integrates these analysis results to generate and update a detailed user preference model. Based on the preference model, the server searches the database for the most suitable artworks. From the search results, the server selects multiple candidates and generates comments explaining why each artwork is the best fit for the user.

[1599] The device displays a list of artworks and explanatory comments to the user. The user reviews the suggested artworks and selects the one they like. Detailed information about the selected artwork and a purchase link are displayed, allowing the user to proceed with the purchase. If the user does not purchase the artwork, they can also enter feedback about the artwork. The device sends the feedback or purchase information to the server, and the server receives the feedback and updates the user's preference data.

[1600] As a concrete example, the following scenario can be considered.

[1601] Example: Art suggestions based on tastes and emotional states

[1602] 1. User: Access the system and select "Art to watch when you want to relax."

[1603] 2. On the device, the user is prompted to upload photos and music to relax with.

[1604] 3. User: Upload photos of the music you like to listen to when relaxing or the interior design.

[1605] 4. Server: Analyzes the uploaded data and evaluates the color and mood of the music.

[1606] 5. Emotion engine: Analyzes the user's current emotional state and sends it to the server as a "want to relax" state.

[1607] 6. Server: Using the collected information, search the database for suitable artworks for relaxation.

[1608] 7. Server: Select multiple candidates from the search results and write a description of each artwork explaining why it is suitable for the user.

[1609] 8. Terminal: Presents the user with a list of candidate artworks and descriptions.

[1610] 9. User: Selects the artwork they like and proceeds to checkout.

[1611] In this way, the system takes into account the user's tastes, interior style, and emotional state to suggest the most suitable artwork, deepening their understanding of art and encouraging them to make purchases.

[1612] The processing flow will be explained below.

[1613] Step 1:

[1614] A user logs into the system using a terminal.

[1615] Step 2:

[1616] The device displays a user registration form when used for the first time.

[1617] Step 3:

[1618] The user enters basic information such as name, email address, hobbies, interior style, and budget.

[1619] Step 4:

[1620] The terminal transmits the input data to the server.

[1621] Step 5:

[1622] The server stores the received data in a database.

[1623] Step 6:

[1624] The server generates questions for the user based on the results of the initial survey.

[1625] Step 7:

[1626] The device interactively asks the user questions (e.g., "What is your favorite genre of art?").

[1627] Step 8:

[1628] The user enters the answer.

[1629] Step 9:

[1630] The device sends the response to the server.

[1631] Step 10:

[1632] The server analyzes the responses and updates the user's preference data.

[1633] Step 11:

[1634] The device prompts the user to input images, videos, music, and emotional states using an emotion engine.

[1635] Step 12:

[1636] Users upload photos, videos, music and emotional information.

[1637] Step 13:

[1638] The terminal transmits the uploaded data to the server.

[1639] Step 14:

[1640] The server runs the uploaded photos through image analysis algorithms to evaluate color and style.

[1641] Step 15:

[1642] The server analyzes the music and video data and extracts genres and themes.

[1643] Step 16:

[1644] The emotion engine analyzes the user's emotional state from their facial expressions and voice and sends the data to the server.

[1645] Step 17:

[1646] The server integrates the analysis results (images, music, videos, emotions) to generate and update a detailed preference model of the user.

[1647] Step 18:

[1648] The server searches the database for the most suitable artwork based on the preference model.

[1649] Step 19:

[1650] The server generates a list of artworks as search results.

[1651] Step 20:

[1652] The server generates comments explaining why each artwork is suitable for the user.

[1653] Step 21:

[1654] The terminal displays the generated list of artworks and explanatory comments to the user.

[1655] Step 22:

[1656] The user reviews the proposed artworks and selects the one they like best.

[1657] Step 23:

[1658] Your device will display detailed information and a link to purchase the selected title.

[1659] Step 24:

[1660] The user completes a purchase or provides feedback about the work.

[1661] Step 25:

[1662] The device sends feedback or purchase information to the server.

[1663] Step 26:

[1664] The server receives the feedback and updates the user's preference data.

[1665] This series of steps suggests the most suitable artworks based on the user's tastes, interior style, and even emotional state, helping them better understand art and encourage them to make purchases.

[1666] Example 2

[1667] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1668] Conventional art suggestion systems have the problem of not being able to suggest the most suitable artworks because they do not fully consider the user's preferences or emotional state. In addition, they lack a system to reflect changes in user preferences, making it difficult to respond to individual needs.

[1669] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through a questionnaire, means for analyzing image, music, or video media data, means for analyzing the user's emotional state, means for integrating the analysis results to generate and update a user preference model, means for searching a database for an optimal visual artwork based on the preference model, and means for presenting the searched visual artwork to the user and explaining why the artwork is optimal. This makes it possible to suggest optimal artworks that take the user's preferences and emotional state into consideration.

[1670] "User preference information and interior style information" is information related to the user's hobbies and preferences, and the interior design style of the user's home, office, etc.

[1671] A "survey" is a set of questions distributed to users to understand their detailed hobbies and preferences.

[1672] "Image, music or video media data" refers to image files, music files and video files uploaded by users for relaxation or related to their hobbies.

[1673] "User's emotional state" is information that indicates the user's current emotional or psychological state.

[1674] "Means for integrating analysis results to generate and update a user preference model" refers to the process of analyzing collected data to create and update an individual preference model that reflects the user's tastes and emotional state.

[1675] The "means for retrieving optimal visual artworks from a database based on a preference model" is a process for retrieving optimal artworks from a database using a user's preference model.

[1676] "A means of presenting visual artworks resulting from a search result to a user and explaining why each artwork is the most appropriate" is a process of presenting artworks retrieved from a database to a user and explaining why each artwork is the most appropriate for the user.

[1677] "Feedback Information" refers to the opinions and ratings you provide regarding the artworks you select or purchase.

[1678] The present invention is a system that performs detailed analysis based on a user's preference information and interior style information to suggest the most suitable artwork. The system mainly includes a server, a terminal, and an emotion engine. The user accesses the system through the terminal.

[1679] System configuration

[1680] 1. Means of collecting user preference information and interior style information

[1681] Device: When using for the first time, a user registration form is displayed and the user enters basic information such as name, email address, hobbies, interior style, and budget.

[1682] Terminal: Sends the entered information to the server.

[1683] Server: Receives user information and stores it in a database.

[1684] 2. A means of analyzing detailed user preferences through surveys

[1685] Server: Generates initial survey questions based on user information.

[1686] Terminal: Questions are presented to the user interactively, and the user answers them.

[1687] Terminal: Sends survey responses to the server.

[1688] Server: Analyzes the received survey responses and updates the user preference data.

[1689] 3. Means for analyzing image, music or video media data

[1690] Terminal: Provides an interface for users to upload photos of their favorite music and interior design when relaxing.

[1691] User: Upload media data such as images, music, and videos.

[1692] Terminal: Sends the uploaded data to the server.

[1693] Server: Analyzes the received image data using an image analysis algorithm (e.g., OpenCV).

[1694] Server: Analyzes music and video data using a library (e.g., LIBROSA) to extract genres and themes.

[1695] 4. A means of analyzing the user's emotional state

[1696] Emotion engine: Analyzes the user's current emotional state from their facial expressions and voice, and sends that information to the server.

[1697] 5. Means of integrating the analysis results to generate and update user preference models

[1698] Server: Integrates the analysis results of media data and emotional information to generate and update a detailed user preference model.

[1699] 6. A method for searching a database for suitable visual artworks based on a preference model

[1700] Server: Search for visual artworks from a database using the preference model (e.g., using Elasticsearch).

[1701] 7. A way to present users with visual art results and explain why they are the best.

[1702] Server: Selects candidate works from the search results and uses a generative AI model to generate comments explaining why each one is the best.

[1703] Terminal: Displays a list of artworks and explanatory comments to the user.

[1704] Specific examples

[1705] Example: Art suggestions based on tastes and emotional states

[1706] 1. User: Access the system and select "Art to watch when you want to relax."

[1707] 2. On the device, the user is prompted to upload photos and music to relax with.

[1708] 3. User: Upload photos of the music you like to listen to when relaxing or the interior design.

[1709] 4. Server: Analyzes the uploaded data and evaluates the color and musical atmosphere (using OpenCV for image data and LIBROSA for music data).

[1710] 5. Emotion engine: Analyzes the user's current emotional state and sends it to the server as a "want to relax" state.

[1711] 6. Server: Based on the collected information, search the database for suitable artworks for relaxation (using Elasticsearch).

[1712] 7. Server: Selects multiple candidates from the search results and uses a generative AI model to compile a description of why each artwork is suitable for the user.

[1713] 8. Terminal: Presents the user with a list of candidate artworks and descriptions.

[1714] 9. User: Selects the artwork they like and proceeds to checkout.

[1715] Prompt Sentence Examples

[1716] plain text

[1717] Describe art suggestions based on user preferences, interior style, and emotional state. Consider the following:

[1718] User basic information (name, email address, hobbies, interior style, budget)

[1719] User-uploaded media data (images, music, videos)

[1720] Emotion engine analysis results

[1721] Pick the best piece of art and explain why it's the best fit for your users.

[1722] In this way, the system of the present invention can effectively suggest the most suitable artworks by reflecting the preferences and emotional state of each user in detail.

[1723] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1724] Step 1:

[1725] A user accesses the system and logs in by entering their name and password in the login form. When using the system for the first time, the terminal displays a user registration form, and the user enters basic information such as their name, email address, hobbies, interior style, and budget. The entered information is sent from the terminal to the server.

[1726] Input: Basic information such as name, email address, hobbies, interior style, budget, etc.

[1727] Output: Basic information about the user sent to the server

[1728] Step 2:

[1729] The server receives the user's basic information sent from the device and stores it in a database, which is used for subsequent analysis.

[1730] Input: Basic user information sent from the device

[1731] Output: Basic user information stored in the database

[1732] Step 3:

[1733] The server generates initial survey questions based on the user information, and sends the generated questions to the terminal, which then displays the survey questions to the user in an interactive format. The user answers the questions.

[1734] Input: Basic user information stored in the database

[1735] Output: Generated initial survey questions

[1736] Step 4:

[1737] The user answers the questionnaire and enters the answers into the terminal, which then sends the user's answers to the server.

[1738] Input: User's survey response

[1739] Output: Survey responses sent to the server

[1740] Step 5:

[1741] The server analyzes the received survey responses and updates the user's preference data using a text analysis algorithm.

[1742] Input: User's survey response

[1743] Output: Updated user preference data

[1744] Step 6:

[1745] The device provides a user with an interface that prompts the user to upload media data such as images, videos, and music, and input emotional information. The user uploads relevant media data and inputs emotional information, and these data are transmitted from the device to the server.

[1746] Input: User-uploaded media data and emotional information

[1747] Output: Media data and emotion information sent to the server

[1748] Step 7:

[1749] The server analyzes the received image data using an image analysis algorithm (e.g., OpenCV). Music and video data is analyzed using an analysis library (e.g., LIBROSA) to extract genres and themes. The emotion engine also analyzes the current emotional state from facial expressions and voice, and this information is sent to the server.

[1750] Input: image data, music data, video data, emotional information

[1751] Output: Analysis results (color, style, genre, theme, emotional state)

[1752] Step 8:

[1753] The server integrates the analysis results and generates and updates a detailed user preference model using data fusion techniques.

[1754] Input: Analysis results

[1755] Output: User preference model

[1756] Step 9:

[1757] The server searches the database for the most suitable visual artworks based on the preference model, using a database search engine (e.g., Elasticsearch).

[1758] Input: User preference model

[1759] Output: Searched visual artwork candidates

[1760] Step 10:

[1761] The server selects candidate artworks from the search results and uses a generative AI model to generate comments explaining why each is the best fit. The generated comments and list of artworks are then sent to the device.

[1762] Input: Search results for visual artworks

[1763] Output: Generated explanatory comments and artwork list

[1764] Step 11:

[1765] The device displays the generated explanatory comments and a list of artworks to the user.

[1766] Input: Generated descriptive comments and artwork list

[1767] Output: A list of artworks and explanatory comments displayed to the user

[1768] Step 12:

[1769] The user reviews the proposed artworks and selects the one they like. Detailed information and a link to purchase the selected artwork are displayed. If the user does not purchase, they can enter their opinion through a feedback form. The feedback or purchase information is sent from the device to the server.

[1770] Input: User's work selection, feedback or purchase information

[1771] Output: Feedback or purchase information sent to the server

[1772] Step 13:

[1773] The server receives the feedback or purchase information and updates the user's preference data again.

[1774] Input: Feedback or Purchase Information

[1775] Output: Updated user preference data

[1776] (Application example 2)

[1777] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1778] In conventional logistics centers, real-time analysis of worker efficiency and emotional state is not adequately performed, and optimal work instructions are not provided according to the state of each worker, resulting in problems such as reduced work efficiency and mistakes due to overwork.In addition, dynamic task adjustment based on employee feedback is not performed, making it difficult to optimize the entire operation.

[1779] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user preference information and interior style information, means for analyzing the user's detailed preferences through analysis of images, videos, or music, means for generating and updating a user preference model, means for searching a database for suitable artworks based on the preference model, means for presenting the searched artworks to the user and explaining why the artworks are suitable, and means for analyzing the user's work efficiency and emotional state using a smart device and providing optimal task instructions. This makes it possible to dynamically adjust work instructions in real time and provide optimal instructions according to the status of each worker.

[1780] "User preference information" is information that reflects the user's hobbies, preferences, and interests.

[1781] "Interior style information" is information about the interior decoration of a user's home or office.

[1782] "Image, video or music analysis" refers to the analysis of these media data to assess a user's preferences and emotional state.

[1783] "Detailed user preferences" refers to information about a user's specific hobbies and preferences, extracted from the analyzed data.

[1784] A "user preference model" is a data model that reflects a user's preferences and is generated based on the user's collected preference information.

[1785] The "optimal artwork" is the artwork selected based on the user's preference model and determined to be most suitable for the user.

[1786] A "smart device" is an advanced device that can connect to the Internet and is equipped with various sensors, and examples include smart glasses.

[1787] "Work efficiency" refers to the efficiency and productivity of workers when performing their work.

[1788] "Emotional state" refers to the user's mental state or emotions, including stress, fatigue, joy, etc.

[1789] "Optimal task instructions" are instructions that are most suitable for each task, based on the user's work efficiency and emotional state.

[1790] The system of the present invention analyzes user preference information and emotional states to provide optimal task instructions in order to improve work efficiency and the emotional state of workers in logistics centers. The system mainly consists of a server, terminals, and smart devices (e.g., smart glasses).

[1791] First, the worker (user) wears a smart device and accesses the system through that device. At this time, the user's preference information and interior style information are collected and sent to the server. The server stores this information in a database.

[1792] The server then analyzes media data such as images, videos, and music to assess the user's detailed preferences. This analysis is performed using software libraries such as OpenCV and NLTK. To analyze the user's emotional state, sensors on the smart device are used to analyze facial expressions and voice. OpenCV is used for facial expression analysis, and NLTK, a natural language processing technology, is used for voice analysis.

[1793] Based on the analysis of the user's detailed preference data and emotional state, the server generates and updates a user preference model. Based on the preference model, the server searches for the most suitable artwork from the database and presents it to the user. At the same time, the server analyzes the user's work efficiency and emotional state in real time via the smart device and provides appropriate task instructions.

[1794] As a concrete example, consider the case where an employee at a logistics center uses smart glasses. If the smart glasses detect high stress levels in the employee through facial expression analysis while they are working, the server sends an instruction message to the smart glasses saying, "Perform light work." After the work is completed, feedback from the employee is collected and stored in a database. This feedback is reflected in the next task instructions.

[1795] An example of a prompt sentence is as follows:

[1796] "Build a system that analyzes the emotional state of logistics center employees and adjusts their workload via smart glasses. Specifically, this would include a function that uses facial expression recognition and emotion analysis to determine stress and fatigue, and provides appropriate work instructions in real time."

[1797] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1798] Step 1:

[1799] The user puts on a smart device (smart glasses) and logs into the system. After logging in, the first time they use the system, they are prompted to enter their preferences (hobbies, interior style, budget, etc.). The information entered by the user is sent from the device to the server and stored in a database.

[1800] Input: User preferences (hobbies, interior style, budget, etc.)

[1801] Output: User preferences stored in a database

[1802] Step 2:

[1803] The server generates an initial questionnaire based on the user's preference information and interior style information, and sends the questions to the terminal. The terminal displays the questions to the user in an interactive format and receives the answers.

[1804] Input: User preference information and interior style information

[1805] Output: User's survey responses

[1806] Step 3:

[1807] The user's response data is sent from the device to a server and analyzed to build detailed user preference data. The user also allows the camera for facial expression analysis and microphone for voice analysis to collect emotional state data.

[1808] Input: User survey responses and emotional data

[1809] Output: Detailed preference and sentiment data

[1810] Step 4:

[1811] The server uses OpenCV and NLTK libraries for facial expression and speech analysis, which analyzes camera images and audio samples to assess the user's emotional state (e.g., stress level).

[1812] Input: Camera images and audio samples

[1813] Output: Parsed emotion data (e.g., stress level)

[1814] Step 5:

[1815] Based on the analysis results, the server generates and updates the user's preference model and stores it in the database.Then, based on the preference model, the server searches the database for the most suitable artworks and recommended tasks based on the questionnaire.

[1816] Input: Parsed emotion data and detailed preference data

[1817] Output: Updated preference model and best artwork or recommended work

[1818] Step 6:

[1819] The server then provides the user with real-time task instructions through the smart glasses, tailored to their emotional state. For example, if high stress is detected, instructions for lighter tasks will be displayed.

[1820] Input: Emotional state and preference model

[1821] Output: Task instructions displayed on smart glasses

[1822] Step 7:

[1823] After completing the task, the device collects feedback from the user and sends it to the server, which uses this information to further update the preference model and reflect it in the next task instructions.

[1824] Input: User feedback

[1825] Output: Updated preference model and next task instructions

[1826] In this way, the system of the present invention can provide optimal work instructions in real time based on the user's preference information and emotional state.

[1827] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1828] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1829] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1830] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1831] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1832] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1833] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1834] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1835] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1836] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1837] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1838] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1839] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1840] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1841] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1842] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1843] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1844] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1845] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1846] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1847] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1848] The following is further disclosed regarding the above embodiment.

[1849] (Claim 1)

[1850] A means for collecting user preference information and interior style information;

[1851] means for analyzing detailed user preferences through image, video or music analysis;

[1852] a means for generating and updating a user preference model;

[1853] A means of searching the database for the most suitable artwork based on the preference model;

[1854] A way to present artworks from search results to users and explain why they are the best.

[1855] A system including:

[1856] (Claim 2)

[1857] 10. The system of claim 1, further comprising means for collecting basic information from the user through an initial questionnaire.

[1858] (Claim 3)

[1859] 10. The system of claim 1, further comprising means for collecting feedback information based on the artwork selected by the user to further update the preference model.

[1860] "Example 1"

[1861] (Claim 1)

[1862] A means for collecting user preference information and interior style information;

[1863] means for analyzing detailed user preferences through image, video or music analysis;

[1864] a means for generating and updating a user preference model;

[1865] A means of searching the database for the most suitable artwork based on the preference model;

[1866] A way to present artworks from search results to users and explain why they are the best.

[1867] a means for collecting user feedback or purchase information and updating the preference model;

[1868] A system including:

[1869] (Claim 2)

[1870] 10. The system of claim 1, further comprising means for collecting basic information from the user through an initial questionnaire.

[1871] (Claim 3)

[1872] 10. The system of claim 1, further comprising means for subjecting the collected images to an analysis algorithm to evaluate color and style.

[1873] "Application Example 1"

[1874] (Claim 1)

[1875] A means for collecting user preference information and interior style information;

[1876] means for analyzing detailed user preferences through image, video or music analysis;

[1877] a means for generating and updating a user preference model;

[1878] A means of searching the database for the most suitable artwork based on the preference model;

[1879] A way to present artworks from search results to users and explain why they are the best.

[1880] A way for users to take pictures of objects in the store and have them suggest the most suitable artwork for the store.

[1881] A system including:

[1882] (Claim 2)

[1883] 10. The system of claim 1, further comprising means for collecting basic information from the user through an initial questionnaire.

[1884] (Claim 3)

[1885] 10. The system of claim 1, further comprising means for collecting feedback information based on the artwork selected by the user to further update the preference model.

[1886] "Example 2: Combining Emotion Engines"

[1887] (Claim 1)

[1888] A means for collecting user preference information and interior style information;

[1889] A means of analyzing detailed user preferences through surveys,

[1890] means for analyzing image, music or video media data;

[1891] a means for analyzing the emotional state of a user;

[1892] a means for integrating the analysis results to generate and update a user preference model;

[1893] a means for searching a database for suitable visual artworks based on the preference model;

[1894] A way to present the visual art results to users and explain why they are the best fit;

[1895] A system including:

[1896] (Claim 2)

[1897] 10. The system of claim 1, further comprising means for generating an initial survey to collect basic information about the user.

[1898] (Claim 3)

[1899] 10. The system of claim 1, further comprising means for collecting feedback information based on the user's selection of visual artwork to further update the preference model.

[1900] "Application example 2 when combining emotion engines"

[1901] (Claim 1)

[1902] A means for collecting user preference information and interior style information;

[1903] means for analyzing detailed user preferences through image, video or music analysis;

[1904] a means for generating and updating a user preference model;

[1905] A means of searching the database for the most suitable artwork based on the preference model;

[1906] A way to present artworks from search results to users and explain why they are the best.

[1907] A method for analyzing a user's work efficiency and emotional state using a smart device and providing optimal task instructions;

[1908] A system including:

[1909] (Claim 2)

[1910] 10. The system of claim 1, further comprising means for collecting basic information from the user through an initial questionnaire.

[1911] (Claim 3)

[1912] 10. The system of claim 1, further comprising means for collecting feedback information based on the artwork selected by the user to further update the preference model.

[1913] (Claim 4)

[1914] 10. The system of claim 1, wherein facial expression analysis and speech analysis are used to assess a user's emotional state and dynamically adjust workload based thereon. [Explanation of symbols]

[1915] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting user preference information and interior style information; means for analyzing detailed user preferences through image, video or music analysis; a means for generating and updating a user preference model; A means of searching the database for the most suitable artwork based on the preference model; A way to present artworks from search results to users and explain why they are the best. A system including:

2. The system of claim 1 further comprising means for collecting basic information from the user through an initial questionnaire.

3. 10. The system of claim 1, further comprising means for collecting feedback information based on user-selected artworks to further update the preference model.

Images