System

Abstract

A system is provided.SOLUTION: A system including a means for constructing a community using an avatar in a virtual space and providing an online event or a workshop, a means for analyzing a content of a user's utterance using a natural language and estimating an emotion of the user, a means for immediately responding to a question of the user by an AI curator and providing information related to an artwork, and a means for updating display content based on a movement or an action of the user in a virtual museum.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's busy lifestyles, it is difficult to find time to visit museums, and many people are unable to enjoy the opportunity to appreciate art. Furthermore, physical museums have limited exhibition space and are unable to display all of the artworks. Furthermore, museum visitors often have limited opportunities to interact with other visitors, making it difficult for them to gain a deep art experience or knowledge. New methods are needed to solve these issues and provide more people with an easy-to-use art experience. [Means for solving the problem]

[0005] This invention provides a system for building a community using avatars in a virtual space and offering online events and workshops. The system includes a means for analyzing the user's natural language utterances and estimating their emotions. It also includes a means for an AI curator to instantly respond to the user's questions and provide information about artworks. It also includes a means for updating the displayed content based on the user's movements and actions in the virtual museum. This allows for a deeper art experience, free from geographical and time constraints.

[0006] A "virtual space" is a virtual three-dimensional space constructed using a computer system, which users can access and experience via the Internet.

[0007] An "avatar" is a character or icon that a user controls as their alter ego in a virtual space, and symbolizes the user's existence.

[0008] A "community" is a group of users who come together in a virtual space with a common purpose or interest, interacting and sharing information.

[0009] An "online event" is a gathering or course that takes place in real time over the internet and that participants can access and participate in from remote locations.

[0010] A "workshop" is a practical course or activity based on a specific theme, where participants learn and experience things together.

[0011] A "natural language" is a language that humans use on a daily basis and that is used to communicate through speech and text.

[0012] "Emotion estimation" means analyzing and identifying the emotions a user is feeling based on their words and actions.

[0013] An "AI Curator" is a virtual curator that uses artificial intelligence technology to provide appropriate information in response to user questions.

[0014] A "virtual museum" is a virtual art museum accessible via the Internet, which exhibits a variety of artworks and allows visitors to view them from the comfort of their own home.

[0015] "Updating the display content" means changing the display information in the virtual space in real time in response to the user's operations and actions. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system for providing an art appreciation experience in a virtual space. Specifically, it builds an avatar community and holds online events and workshops. It also includes a function to analyze the user's natural language comments and estimate their emotions. Furthermore, it has a function that allows an AI curator to instantly respond to the user's questions and provide information about the artworks.

[0038] System overview and basic processing

[0039] The virtual museum system works in cooperation with a server and terminals. The server is responsible for data management and processing, while the terminals function as the user interface.

[0040] User authentication and login

[0041] 1. The server accepts the system startup and provides a page for user authentication.

[0042] 2. The user accesses the login screen on the device and enters their username and password.

[0043] 3. The server receives the entered information and authenticates it by checking it against a database.

[0044] 4. The server sends the authentication result to the terminal and notifies it of the success or failure of the login.

[0045] Displaying and manipulating virtual space

[0046] 1. The server sends the virtual museum data to the terminal.

[0047] 2. The device renders the received data and displays the virtual space to the user.

[0048] 3. Users move around the virtual space and view the exhibits.

[0049] Interactive Tours and Sentiment Analysis

[0050] 1. The user enters questions or comments about the artwork in text on the device.

[0051] 2. The server analyzes the content of the speech using a natural language processing algorithm.

[0052] 3. The server infers the user's emotions and intentions and generates an appropriate response.

[0053] 4. The server sends the response to the terminal and displays it to the user.

[0054] Hosting online events and workshops

[0055] 1. The server manages and notifies schedules for online events and workshops.

[0056] 2. The user checks the event information on the device and applies to participate.

[0057] 3. The server manages the participant list and controls participant access when the event starts.

[0058] 4. Users can join the event on their devices and interact with other participants through their avatars.

[0059] Specific examples

[0060] Example 1: User login and virtual space display

[0061] Scenario 1: User A logs in to the virtual museum

[0062] 1. User A accesses the login screen on the device and enters their username and password.

[0063] 2. The server receives User A's authentication information and checks it against the user data in its database.

[0064] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[0065] 4. The device renders the received data and displays the virtual space to User A.

[0066] Example 2: User questions and AI curator responses

[0067] Scenario 2: User B asks about a specific painting

[0068] 1. User B moves through the virtual space and stops in front of a specific painting.

[0069] 2. User B types "What is the background of this picture?" into the text box.

[0070] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms.

[0071] 4. Based on the analysis results, the server generates background information for the painting and returns a response to User B.

[0072] 5. User B reviews the response and learns more about the painting.

[0073] Example 3: Participating in an online workshop

[0074] Scenario 3: User C attends an online workshop

[0075] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[0076] 2. User C checks the event information on their device and applies to participate.

[0077] 3. At the start of the event, the server creates a virtual room and invites participants.

[0078] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[0079] From the above explanation, it can be understood that the present invention provides a virtual museum experience that allows users to easily enjoy the charm of art even in today's busy world. This allows users to enjoy a deep art experience without being restricted by physical constraints.

[0080] The processing flow will be explained below.

[0081] User login process

[0082] Step 1:

[0083] The user operates the device and accesses the login screen, enters their username and password, and clicks the "Login" button.

[0084] Step 2:

[0085] The terminal generates a request to send the entered user name and password to the server, and sends it to the server.

[0086] Step 3:

[0087] The server checks the received username and password against a database to see if the user is valid.

[0088] Step 4:

[0089] The server generates an authentication result and sends a success or failure message to the terminal.

[0090] Step 5:

[0091] The terminal receives the authentication result and displays a login success message to the user, or an error message if authentication fails.

[0092] Displaying and manipulating virtual space

[0093] Step 1:

[0094] The server prepares to send the virtual museum's 3D models and artwork data to users who have successfully logged in.

[0095] Step 2:

[0096] The server sends the virtual museum data to the terminal.

[0097] Step 3:

[0098] The device processes the virtual museum data received using a rendering engine and displays the virtual space on the user's display.

[0099] Step 4:

[0100] Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[0101] Step 5:

[0102] The device detects the user's actions and reflects the corresponding actions (movement, change of viewpoint, selection of specific objects) in the virtual space in real time.

[0103] Interactive Tours and Sentiment Analysis

[0104] Step 1:

[0105] Users stop in front of a particular piece of art and type their question or comment into a text box.

[0106] Step 2:

[0107] The terminal generates a request for transmitting the input text data to the server, and transmits the request to the server.

[0108] Step 3:

[0109] The text data received by the server is analyzed using a natural language processing engine to infer the user's intentions and emotions.

[0110] Step 4:

[0111] The server generates appropriate response data based on the analysis results, including background information about the work, information about the creator, historical context, etc.

[0112] Step 5:

[0113] The server sends the generated response data to the terminal.

[0114] Step 6:

[0115] The terminal displays the received response data to the user, who then checks the detailed information.

[0116] Hosting online events and workshops

[0117] Step 1:

[0118] The server generates data to notify registered users of new online event and workshop schedules.

[0119] Step 2:

[0120] The server sends the notification data to the registered user's terminal.

[0121] Step 3:

[0122] The device displays the received notification to the user, who then checks the event information.

[0123] Step 4:

[0124] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[0125] Step 5:

[0126] The server accepts the participation application and registers it in the participant list.

[0127] Step 6:

[0128] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[0129] Step 7:

[0130] Users use their devices to access virtual rooms and participate in online events and workshops.

[0131] Step 8:

[0132] Users interact with other participants through their avatars, chatting and exchanging opinions.

[0133] Through the above processing steps, the virtual museum system provides users with a realistic art experience and the ability to acquire knowledge.

[0134] Example 1

[0135] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0136] Currently, many people find it difficult to visit museums and galleries due to physical constraints. However, online art appreciation experiences have limitations, and traditional methods do not allow users to enjoy a deep art experience or interact with other viewers. Furthermore, they lack the ability to provide relevant information about artworks and interactive responses based on the user's emotions. Given these factors, a new system is needed to provide a realistic art appreciation experience in a virtual space.

[0137] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0138] In this invention, the server includes means for transmitting a username and password entered from a terminal to the server for user authentication, comparing the username and password with a database and returning the authentication result, means for transmitting data of the virtual space from the server to the terminal and rendering and displaying the received data by the terminal, and means for the server to analyze the text data using a natural language processing engine when a user stops in front of a specific artwork in the virtual space and enters a question or comment, and generate and return an appropriate response. This allows users to enjoy a deep art experience and interact with other viewers without being restricted by physical constraints.

[0139] A "virtual space" is an imaginary space generated on a computer system that users can explore and manipulate as a 3D model.

[0140] An "avatar" is a digital character that users use to represent themselves in a virtual space.

[0141] An "online event" is a virtual event or gathering that takes place in real time over the Internet, allowing participants to share a common experience in a virtual space.

[0142] A "workshop" is an interactive learning or activity space based on a specific theme, where users can learn as participants by interacting with instructors and other participants.

[0143] "Natural language processing" is a technology that allows computers to analyze and understand human language (natural language), and is used to enable machines to understand and appropriately process user statements and input.

[0144] "Sentiment analysis" is a technique for inferring a user's emotional state from their speech or text, and is used to understand the user's current feelings and intentions.

[0145] "AI Curator" is a system based on artificial intelligence technology that acts as a virtual guide, providing instant information in response to user questions.

[0146] "User authentication" is the process of verifying the identity of a user by checking the credentials (username and password) entered by the user to access a system.

[0147] "Rendering" is the process by which a computer visually displays an image or video based on received 3D model data or other graphics data.

[0148] A "natural language processing engine" is a collection of algorithms and software that analyzes a user's text data, understands its content, and responds appropriately.

[0149] This invention is a system for providing an art appreciation experience in a virtual space. Specifically, it builds an avatar community and holds online events and workshops. It also includes a function to analyze the user's natural language comments and estimate their emotions. Furthermore, it has a function that allows an AI curator to instantly respond to the user's questions and provide information about the artworks.

[0150] System Configuration

[0151] This system operates in cooperation with a server and a terminal. The server is responsible for data management and processing, and the terminal functions as a user interface. The main components of this system and their functions are explained below.

[0152] User authentication and login

[0153] The server accepts system startup and provides a user authentication page. The user accesses the login screen on their terminal and enters their username and password. The server receives the entered information and performs authentication by checking it against a database (for example, MySQL (registered trademark) or PostgreSQL). It sends the authentication result to the terminal and notifies them of the success or failure of the login. This process uses the web server Apache (registered trademark) or Nginx.

[0154] Displaying and manipulating virtual space

[0155] The server sends the virtual museum data to the device, which then renders it using libraries such as WebGL and Three.js to display the virtual space to the user. The user can then move around the virtual space using a keyboard, mouse, or touchscreen to view the exhibits.

[0156] Interactive Tours and Sentiment Analysis

[0157] When a user enters a question or comment about an artwork as text on their device, the server analyzes the text data using a natural language processing algorithm (e.g., Google® NLP API, SpaCy, or NLTK). The server infers the user's emotions and intentions and generates an appropriate response. The response can be generated using a generative AI model (e.g., OpenAI® GPT-3®). The response generated by the server is sent to the device and displayed to the user.

[0158] Hosting online events and workshops

[0159] The server manages and notifies schedules for online events and workshops. Users can check event information on their devices and apply to participate. The server manages the participant list and creates a virtual room at the start of the event to invite participants. Users can access the virtual room on their devices and interact with other participants through avatars.

[0160] Specific examples

[0161] The following are specific examples of this system:

[0162] Example 1: User login and virtual space display

[0163] When User A logs in to the virtual museum

[0164] 1. User A accesses the login screen on the device and enters their username and password.

[0165] 2. The server receives User A's authentication information and checks it against the user data in its database.

[0166] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[0167] 4. The device renders the received data and displays the virtual space to User A.

[0168] Example 2: User questions and AI curator responses

[0169] User B asks about a specific painting

[0170] 1. User B moves through the virtual space and stops in front of a specific painting.

[0171] 2. User B types "What is the background of this picture?" into the text box.

[0172] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms.

[0173] 4. The server generates background information for the painting based on the analysis results and returns a response to User B.

[0174] 5. User B reviews the response and learns more about the painting.

[0175] Example 3: Participating in an online workshop

[0176] User C participates in an online workshop

[0177] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[0178] 2. User C checks the event information on their device and applies to participate.

[0179] 3. At the start of the event, the server creates a virtual room and invites participants.

[0180] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[0181] Example prompt sentence:

[0182] "What is the background to this painting?"

[0183] "Tell me about Monet's works."

[0184] "When is the next online event?"

[0185] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0186] Handling user authentication and login

[0187] Step 1:

[0188] The server accepts the system startup and provides a page for user authentication.

[0189] Specific operation: When the server receives a startup signal, it uses a web server (Apache or Nginx) to generate an authentication page and sends the URL to the terminal.

[0190] Input: Start signal

[0191] Output: URL of authentication page

[0192] Step 2:

[0193] The user accesses the login screen on the device and enters their username and password.

[0194] Specific operation: The user uses a browser to access the authentication page displayed on the device, enters their username and password, and clicks the submit button.

[0195] Input: Username, Password

[0196] Output: Sending authentication information

[0197] Step 3:

[0198] The server receives the entered information and authenticates it by checking it against a database.

[0199] Specific operation: The server receives the authentication information sent from the terminal and checks it against the user data in the database (MySQL or PostgreSQL) using an SQL query to confirm that it matches.

[0200] Input: Username, Password

[0201] Output: Authentication result

[0202] Step 4:

[0203] The server sends the authentication result to the terminal, notifying it of the success or failure of the login.

[0204] Specific operation: The server formats the authentication result as an HTTP response and sends it to the terminal. The terminal receives the result and redirects the user to the virtual museum screen if the login is successful, or displays an error message if the login is unsuccessful.

[0205] Input: Authentication result

[0206] Output: Login success message or error message

[0207] Virtual space display and operation processing

[0208] Step 1:

[0209] The server sends the virtual museum data to the terminal.

[0210] Specific operation: The server generates the 3D model data (JSON or XML format) required for the virtual museum and sends it to the terminal.

[0211] Input: Request after successful login

[0212] Output: 3D model data

[0213] Step 2:

[0214] The device renders the received data and displays the virtual space to the user.

[0215] Specific operation: The device renders the received 3D model data using graphics libraries such as WebGL or Three.js, and displays the virtual space on the browser.

[0216] Input: 3D model data

[0217] Output: Display of virtual space

[0218] Step 3:

[0219] Users move around the virtual space and view the exhibits.

[0220] Specific actions: Users navigate the virtual space using a keyboard, mouse, or touchscreen, and can click on specific exhibits to view more information.

[0221] Input: User actions

[0222] Output:Detailed information about the exhibit

[0223] Interactive tour and sentiment analysis processing

[0224] Step 1:

[0225] Users enter questions and comments about the artwork via text on their device.

[0226] Specific behavior: The user enters a question or comment in the text box and clicks the submit button.

[0227] Input: Text data (questions and comments)

[0228] Output: Sending text data

[0229] Step 2:

[0230] The server analyzes the content of the speech using natural language processing algorithms.

[0231] Specific operation: The server receives the text data, analyzes it using a natural language processing engine (Google NLP API, SpaCy, NLTK), and infers the user's emotions and intentions.

[0232] Input: Text data

[0233] Output: Analysis results (emotions and intentions)

[0234] Step 3:

[0235] The server estimates the user's emotions and intentions and generates an appropriate response.

[0236] Specific operation: The server generates an appropriate response using a generative AI model (OpenAI GPT-3) based on the analysis results.

[0237] Input: Analysis results

[0238] Output: Response text

[0239] Step 4:

[0240] The server sends the response to the terminal and displays it to the user.

[0241] Specific operation: The server formats the generated response text as an HTTP response and sends it to the terminal, which then displays the response text to the user.

[0242] Input: Response text

[0243] Output: View the response

[0244] Handling online events and workshops

[0245] Step 1:

[0246] The server manages and notifies schedules for online events and workshops.

[0247] Specific operation: The server stores the schedule of the online event in a database and notifies participating users via email or other means.

[0248] Input: Online event information

[0249] Output: Notification message

[0250] Step 2:

[0251] The user checks the event information on their device and applies to participate.

[0252] Specific operation: The user checks the event information on the device and presses the participation application button.

[0253] Input: Event participation application

[0254] Output: Send application information

[0255] Step 3:

[0256] The server manages the participant list and controls participant access at the start of the event.

[0257] Specific operation: The server manages the participant list, creates a virtual room at the start of the event, and invites participants.

[0258] Input: Participant list

[0259] Output: Virtual room URL

[0260] Step 4:

[0261] Users join events on their devices and interact with other participants through avatars.

[0262] Specific operation: A user accesses the URL of a virtual room and interacts with other participants through their avatars.

[0263] Input: Virtual room URL

[0264] Output: Interaction through avatars

[0265] (Application example 1)

[0266] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0267] Modern retail experiences require users to explore a wide variety of products without physical constraints and to receive real-time information to support purchasing decisions. However, existing online shopping systems require a lot of effort for users to obtain detailed product information and receive appropriate recommendations, and they face challenges in providing personalized responses based on the user's emotions and intent. Furthermore, there is a lack of an environment for effectively hosting new product launches and fashion events online.

[0268] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0269] In this invention, the server includes: a means for building a community using avatars in a virtual space and providing online events and workshops; a means for analyzing users' natural language utterances and estimating their emotions; a means for an AI curator to instantly respond to users' questions and provide information; a means for updating display content based on the user's movements and actions in the virtual store; a means for searching for items in the virtual store in natural language using a smartphone or head-mounted display and displaying the results; a means for making real-time suggestions for recommended items and outfits; and a means for managing and notifying the schedule of online fashion events and workshops. This allows users to freely move around the virtual store using their own avatars, efficiently explore a variety of products, and receive personalized information and recommendations in real time. Furthermore, it also becomes possible to effectively hold new product launches and fashion events online, significantly improving the user experience.

[0270] "Virtual space" refers to a digital environment in which users can operate in a virtual realm that is different from their actual physical environment.

[0271] An "avatar" is a digital character with which a user interacts in a virtual space.

[0272] A "community" is a group of users who share a common purpose or interest and interact with each other.

[0273] An "online event" is a real-time event or occasion held over the Internet.

[0274] A "workshop" is a participatory educational and training setting for learning specific skills and knowledge.

[0275] "Natural language" refers to the language used by humans on a daily basis, which needs to be converted into a form that is easy for computers to understand.

[0276] "Emotion" is a concept that represents a human mental state, such as joy, sadness, or anger.

[0277] "AI Curator" refers to a system that uses artificial intelligence to respond to user questions and provide information.

[0278] "Document" does not necessarily mean text data, but may also include drawings, explanatory text, diagrams, etc.

[0279] "Display content" is the collection of information that a user can visually see on the screen.

[0280] A "smartphone" is a portable information terminal that can use a variety of applications in addition to the functions of a mobile phone.

[0281] A "head-mounted display" is a display device worn on the head, and is primarily used to experience virtual reality and augmented reality.

[0282] "Items" are goods or merchandise that users can view or purchase within the virtual space.

[0283] "Recommendation" refers to the presentation of specific products or information based on the user's preferences and behavior.

[0284] "Coordination suggestions" refers to providing recommendations on how to effectively combine fashion items.

[0285] "Managing your schedule" refers to scheduling and coordinating specific events and tasks.

[0286] "Notification" refers to the sending of a message to a user informing them of important information or the onset of an event.

[0287] This invention is a system that allows users to use avatars in a virtual space to explore products and engage in two-way communication, such as asking questions and sending comments. Specifically, using devices such as smartphones or head-mounted displays, users can access a virtual store, search for and view items in real time, and obtain information about artworks.

[0288] System overview and basic processing

[0289] The virtual fashion boutique system includes a smartphone, a head-mounted display, and a server as its main hardware and software components. Specific software components include a natural language processing engine and a sentiment analysis engine.

[0290] User authentication and login

[0291] The server accepts system startup and provides a page for user authentication. Using a terminal, the user accesses the login screen and enters their username and password. The server receives the entered information and performs authentication by comparing it with a database. If authentication is successful, the server sends the virtual store data to the terminal and notifies it that login was successful.

[0292] Displaying and manipulating virtual space

[0293] The server sends the virtual boutique data to the device, which then renders the received data and displays the virtual space to the user, allowing the user to move around the virtual space and view the exhibited products.

[0294] Natural Language Search and Sentiment Analysis

[0295] Users input questions and opinions about fashion items via text. The device sends the input data to the server, which then analyzes the text using a natural language processing engine. The server then infers the user's emotions and intentions, generates an appropriate response, and sends it to the device.

[0296] Display of product information and AI curator response

[0297] When a user stops in front of a specific item, the server generates detailed information related to that item and displays it to the user. The AI ​​curator instantly responds to the user's questions and provides relevant product information.

[0298] Hosting online events and workshops

[0299] The server schedules new product launches and fashion events and notifies users. Users use their devices to check event information and register for participation. The server manages the participant list and controls participant access when the event starts. The device displays the event in real time and allows users to interact with other participants.

[0300] Specific examples

[0301] Example 1: User login and virtual space display

[0302] Scenario 1: A user logs into a virtual fashion boutique

[0303] The user accesses the login screen on their smartphone and enters their username and password. The server receives the user's authentication information and compares it with the user data in the database. If authentication is successful, the server sends the virtual boutique data to the device along with a "Login successful" message. The device then renders the received data and displays the virtual space to the user.

[0304] Example 2: User questions and AI curator responses

[0305] Scenario 2: User asks about a specific dress

[0306] The user moves through the virtual space and stops in front of a particular dress. The user types "What is the trend for this dress?" into a text box. The device sends the input data to the server, which analyzes the text using a natural language processing engine. Based on the analysis results, the server generates trend information for the dress and returns a response to the user. The user checks the response and deepens their knowledge about the dress.

[0307] Example 3: Participating in an online fashion event

[0308] Scenario 3: User attends an online fashion event

[0309] The server notifies users of the schedule for a new online fashion event, "Introducing the Latest Trends." Users check the event information on their devices and apply to participate. When the event starts, the server creates a virtual room and invites participants. Users access the virtual room and participate in the event while interacting with other participants.

[0310] Prompt Sentence Examples

[0311] A user logs into a fashion boutique and asks, "Tell me about the latest dress trends." An AI curator then answers the question, and the user is invited to a new fashion event.

[0312] With the above-described configuration, the present invention realizes a virtual store where a wide variety of products can be efficiently and effectively searched for without being restricted by physical movement or time.

[0313] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0314] Step 1:

[0315] The server accepts the system startup and provides a page for user authentication.

[0316] Input: User's access request

[0317] Data calculation: Display of user authentication page

[0318] Output:Login screen

[0319] Step 2:

[0320] The user accesses the login screen on the device and enters their username and password.

[0321] Input: Username, Password

[0322] Data Computation: Input Data Collection and Formatting

[0323] Output: Authentication data

[0324] Step 3:

[0325] The server receives the entered information and authenticates it by checking it against a database.

[0326] Input: Authentication data

[0327] Data calculation: database matching, authentication calculation

[0328] Output: Authentication result

[0329] Step 4:

[0330] The server sends the authentication result to the terminal, notifying it of the success or failure of the login.

[0331] Input: Authentication result

[0332] Data Calculation: Message Generation

[0333] Output: Notification of successful or failed login

[0334] Step 5:

[0335] The server sends the virtual boutique data to the terminal.

[0336] Input: Login success notification

[0337] Data calculation: data packaging, transmission processing

[0338] Output: Virtual boutique data

[0339] Step 6:

[0340] The device renders the received data and displays the virtual space to the user.

[0341] Input: Virtual Boutique Data

[0342] Data calculation: data rendering, graphics processing

[0343] Output: Virtual space display

[0344] Step 7:

[0345] Users move around the virtual space and view the exhibits.

[0346] Input: User Action

[0347] Data calculation: Position update, display update

[0348] Output: Updated display

[0349] Step 8:

[0350] The user enters a text question about a particular item.

[0351] Input: User text input

[0352] Data calculation: Collecting and structuring input data

[0353] Output: Question data

[0354] Step 9:

[0355] The terminal sends the input data to the server.

[0356] Input: Question data

[0357] Data calculation: Data transmission processing

[0358] Output: Data sent

[0359] Step 10:

[0360] The server analyzes the text using a natural language processing engine.

[0361] Input: Data sent

[0362] Data Computing: Natural Language Analysis, Text Semantic Understanding

[0363] Output: Analysis results

[0364] Step 11:

[0365] The server estimates the user's emotions and intentions, generates an appropriate response, and sends it to the device.

[0366] Input: Analysis results

[0367] Data calculation: emotion estimation, response generation

[0368] Output: Generated response data

[0369] Step 12:

[0370] When a user stops in front of a particular item, the server generates and displays detailed information related to that item.

[0371] Input: Item stop event

[0372] Data Calculations: Finding and Generating More Information

[0373] Output: Show detailed information

[0374] Step 13:

[0375] The server manages the schedule of new product launches and fashion events and notifies users.

[0376] Input: Event schedule information

[0377] Data calculation: Schedule management, notification generation

[0378] Output: Notification message

[0379] Step 14:

[0380] The user checks the event information on the device and applies to participate.

[0381] Input: Event participation application

[0382] Data processing: collection and transmission of application data

[0383] Output: Participation application data

[0384] Step 15:

[0385] The server manages the participant list and controls participant access at the start of the event.

[0386] Input: Application data

[0387] Data operations: list management, access control

[0388] Output: Participant access control

[0389] Step 16:

[0390] The device displays the event in real time and allows users to interact with other participants.

[0391] Input: Real-time event data

[0392] Data calculation: Event display and AC function provided

[0393] Output: Real-time event display, interaction function

[0394] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0395] This invention relates to a system for providing an art appreciation experience in a virtual space, and in particular to a combined emotion engine that recognizes user emotions. Specifically, it includes functions for building an avatar community and hosting online events and workshops, analyzing users' natural language utterances to estimate their emotions, providing information about artworks through an AI curator that responds to users' questions in real time, and updating the display content based on the user's movements and actions in the virtual museum.

[0396] System overview and basic processing

[0397] The virtual museum system works in conjunction with a server and terminals. The server is responsible for data management and processing, while the terminals function as the user interface. Furthermore, an emotion engine is built in, which analyzes user emotion data and provides feedback to the entire system.

[0398] User authentication and login

[0399] 1. The server accepts the system startup and provides a page for user authentication.

[0400] 2. The user accesses the login screen on the device and enters their username and password.

[0401] 3. The server receives the entered information and authenticates it by checking it against a database.

[0402] 4. The server sends the authentication result to the terminal and notifies it of the success or failure of the login.

[0403] Displaying and manipulating virtual space

[0404] 1. The server sends the virtual museum's 3D model and artwork data to the user who has successfully logged in.

[0405] 2. The device processes the received data using a rendering engine and displays the virtual space on the user's display.

[0406] 3. Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[0407] Interactive Tours and Sentiment Analysis

[0408] 1. Users stop in front of a particular piece of art and type their question or comment into a text box.

[0409] 2. The terminal sends the entered text data to the server.

[0410] 3. The server analyzes the received text data using a natural language processing engine and uses an emotion engine to infer the user's intentions and emotions.

[0411] 4. The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[0412] 5. The server generates a response and sends it to the terminal, where it displays it to the user.

[0413] Hosting online events and workshops

[0414] 1. The server notifies registered users of new online events and workshops scheduled.

[0415] 2. The device displays the received notification to the user, and the user checks the event information.

[0416] 3. If the user wishes to register, he / she sends a participation application request to the server from his / her terminal.

[0417] 4. The server accepts the participation application and registers it on the participant list.

[0418] 5. At the start of the event, the server creates a virtual room for participants and sends them an access link.

[0419] 6. Users use their devices to access the virtual room and participate in online events and workshops.

[0420] 7. Users can interact with other participants through their avatars, chat and exchange opinions.

[0421] Specific examples

[0422] Example 1: User login and sentiment engine analysis

[0423] Scenario 1: User A logs in to the virtual museum

[0424] 1. User A accesses the login screen on the device and enters their username and password.

[0425] 2. The server receives User A's authentication information and checks it against the user data in its database.

[0426] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[0427] 4. The device renders the received data and displays the virtual space to User A.

[0428] Example 2: User Questions and Emotion Engine Responses

[0429] Scenario 2: User B asks about a specific painting

[0430] 1. User B moves through the virtual space and stops in front of a specific painting.

[0431] 2. User B types "What is the background of this picture?" into the text box.

[0432] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms and an emotion engine.

[0433] 4. Based on the analysis results, the server generates customized information based on the painting's background information and the user's emotions, and returns a response to User B.

[0434] 5. User B reviews the response and learns more about the painting.

[0435] Example 3: Online workshops and the use of sentiment analysis

[0436] Scenario 3: User C attends an online workshop

[0437] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[0438] 2. User C checks the event information on their device and applies to participate.

[0439] 3. At the start of the event, the server creates a virtual room and invites participants.

[0440] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[0441] 5. The server analyzes users' comments in real time during the workshop using an emotion engine and provides interactive content that responds to participants' emotional changes.

[0442] As explained above, by combining an emotion engine that recognizes the user's emotions, the present invention can provide a more personalized art experience and deepen the user's appreciation experience, allowing the user to enjoy a deeper art experience without being restricted by physical constraints.

[0443] The processing flow will be explained below.

[0444] User login process

[0445] Step 1:

[0446] The user operates the device and accesses the login screen, enters their username and password, and clicks the "Login" button.

[0447] Step 2:

[0448] The terminal generates a request to send the entered user name and password to the server, and sends it to the server.

[0449] Step 3:

[0450] The server checks the received username and password against a database to see if the user is valid.

[0451] Step 4:

[0452] The server generates an authentication result and sends a success or failure message to the terminal.

[0453] Step 5:

[0454] The terminal receives the authentication result and displays a login success message to the user, or an error message if authentication fails.

[0455] Displaying and manipulating virtual space

[0456] Step 1:

[0457] The server prepares to send the virtual museum's 3D models and artwork data to users who have successfully logged in.

[0458] Step 2:

[0459] The server sends the virtual museum data to the terminal.

[0460] Step 3:

[0461] The device processes the virtual museum data received using a rendering engine and displays the virtual space on the user's display.

[0462] Step 4:

[0463] Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[0464] Step 5:

[0465] The device detects the user's actions and reflects the corresponding actions (movement, change of viewpoint, selection of specific objects) in the virtual space in real time.

[0466] Interactive Tours and Sentiment Analysis

[0467] Step 1:

[0468] Users stop in front of a particular piece of art and type their question or comment into a text box.

[0469] Step 2:

[0470] The terminal generates a request for transmitting the input text data to the server, and transmits the request to the server.

[0471] Step 3:

[0472] The text data received by the server is analyzed using a natural language processing engine, and the emotion engine is used to infer the user's intentions and emotions.

[0473] Step 4:

[0474] The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[0475] Step 5:

[0476] The server sends the generated response data to the terminal.

[0477] Step 6:

[0478] The terminal displays the received response data to the user, who then checks the detailed information.

[0479] Hosting online events and workshops

[0480] Step 1:

[0481] The server generates data to notify registered users of new online event and workshop schedules.

[0482] Step 2:

[0483] The server sends the notification data to the registered user's terminal.

[0484] Step 3:

[0485] The device displays the received notification to the user, who then checks the event information.

[0486] Step 4:

[0487] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[0488] Step 5:

[0489] The server accepts the participation application and registers it in the participant list.

[0490] Step 6:

[0491] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[0492] Step 7:

[0493] Users use their devices to access virtual rooms and participate in online events and workshops.

[0494] Step 8:

[0495] Users interact with other participants through their avatars, chatting and exchanging opinions.

[0496] Step 9:

[0497] During the workshop, the server analyzes users' comments in real time using an emotion engine and provides interactive content that responds to participants' emotional changes.

[0498] Through the above processing steps, the virtual museum system provides users with an immersive art experience and knowledge acquisition, and by combining it with an emotion engine, it can realize a personalized experience based on the user's emotions.

[0499] Example 2

[0500] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0501] Virtual art appreciation lacks a personalized experience that takes into account the user's emotions. This can lead to a decrease in user satisfaction and motivation to participate. Furthermore, while users need fast and accurate responses to their questions, existing systems lack immediacy and adaptability.

[0502] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for building a community using avatars in a virtual space and providing online events and workshops; a means for analyzing the content of users' natural language utterances and estimating the user's emotions; a means for an AI curator to instantly respond to users' questions and provide information about artworks; a means for updating display content based on the user's movements and actions in the virtual museum; a means for performing user authentication and login processing; a means for displaying the virtual space using a user interface rendering engine; a means for analyzing input text data using a natural language processing engine and estimating the user's intentions and emotions using an emotion engine; and a means for generating response data based on the analysis results and displaying it to the user. This enables a personalized art appreciation experience that takes user emotions into consideration, thereby improving user satisfaction and willingness to participate. It also enables rapid and accurate responses, providing a more interactive and engaging experience.

[0503] A "virtual space" is a virtual environment generated using computers and virtual reality technology that users can experience through their senses of sight, hearing, touch, etc.

[0504] An "avatar" is a character or alter ego that a user uses to represent themselves in a virtual space.

[0505] A "community" is a group of users who come together under a common purpose or interest and interact and cooperate with each other.

[0506] An "online event" is a gathering or event that takes place in real time over the internet.

[0507] A "workshop" is an event held for participants to learn about a particular topic or technology and share practical experience.

[0508] "User authentication" means the process of verifying that a user attempting to access a system is a registered and authorized user.

[0509] A "natural language processing engine" is a computer algorithm or program that analyzes, understands, and generates the natural language used by humans on a daily basis.

[0510] An "emotion engine" is an algorithm or program that analyzes and infers a user's emotions from their text or voice data.

[0511] An "artificial intelligence curator" is a system that uses AI technology to provide information in response to user questions and operations, and to explain artworks.

[0512] A "virtual museum" is a museum in a virtual space set up on the Internet, where digital artworks and exhibits can be viewed.

[0513] A "rendering engine" is a program or software that generates computer graphics and displays them on a display.

[0514] A "session" refers to a series of operations and communications between when a user logs in to the system and when they log out, and is a temporary unit of communication that is specific to each user.

[0515] This invention relates to a system for providing an art appreciation experience in a virtual space, particularly incorporating an emotion engine that recognizes the user's emotions. Users can access the virtual space using an avatar to view exhibits and participate in online events and workshops. It also has the ability to analyze the user's comments and generate appropriate responses based on their emotions.

[0516] System configuration

[0517] This system works in conjunction with a server and a device. The server manages and processes data, while the device functions as the user interface. It is equipped with an emotion engine and a natural language processing engine, which analyzes user emotions to provide a personalized experience. Specifically, it includes the following components:

[0518] 1. Server

[0519] Database Management System (DBMS): Manages user information and artwork data. For example, MySQL or PostgreSQL can be used.

[0520] Natural language processing engine: Analyzes user utterances. For example, GPT-4 (registered trademark) is used.

[0521] Emotion engine: Estimates user emotions.

[0522] Rendering engine: Processes the 3D models to generate the virtual space. For example, Unity or Unreal Engine is used.

[0523] 2. Terminal

[0524] User Interface (UI): Provides access to and control of virtual spaces, including PCs and VR devices.

[0525] Input devices: keyboard, mouse, VR controller, etc.

[0526] 3. Users

[0527] Avatar: A character or alter ego that a user controls in a virtual space.

[0528] Specific processing details

[0529] User authentication and login

[0530] The server starts the system and provides a user authentication page. The user enters their username and password on the terminal and sends them to the server. The server checks the database for authentication and returns the result to the terminal.

[0531] Displaying and manipulating virtual space

[0532] After a successful login, the server sends data on the virtual museum's 3D model and artworks. The device receives the data and renders it using a rendering engine. The user then uses a mouse or VR controller to move around the virtual space.

[0533] Interactive Tours and Sentiment Analysis

[0534] Users stop in front of a particular artwork and enter their questions or comments in text. The device sends the data to a server, where it is analyzed by a natural language processing engine. The emotion engine infers the user's intentions and emotions, and the server generates an appropriate response and sends it to the device.

[0535] Hosting online events and workshops

[0536] The server notifies users of new event and workshop schedules. Users can check the schedules on their devices and apply to participate. The server registers participants on a list and opens a virtual room at the start of the event. Users can access the room and interact with other participants.

[0537] Specific scenarios

[0538] Scenario 1: User Logged In

[0539] User A accesses the login screen on their device and enters their username and password. The server receives the authentication information and checks it against the database. If authentication is successful, the virtual museum data is sent to the device and the virtual space is displayed to User A.

[0540] Scenario 2: User Questions and Sentiment Analysis

[0541] User B asks a question about a particular painting. He types, "What is the background of this painting?" into his device. The text data is sent to the server, where it is analyzed by the natural language processing engine and emotion engine. The server generates a response based on the analysis results and sends it to the device.

[0542] Scenario 3: Participating in an online workshop

[0543] User C receives the schedule for a new online workshop and applies to participate. The server registers the participants in the list and generates a virtual room at the start of the event. User C accesses the room and participates in the workshop.

[0544] Prompt Sentence Examples

[0545] "When a user stands in front of a particular painting and asks, 'What is the background to this painting?', provide context for the artwork based on the user's emotions."

[0546] Through these functions, the present invention can recognize the user's emotions and provide a personalized art appreciation experience, allowing the user to enjoy a deep art experience without being restricted by physical constraints.

[0547] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0548] The flow of this system's program processing

[0549] User authentication and login

[0550] Step 1:

[0551] The server accepts the system startup, provides a user authentication page, generates a URL for the user to access the login screen, and returns it as an HTTP response.

[0552] Input: User's access request

[0553] Output: User authentication page

[0554] Specific behavior:

[0555] The server receives the HTTP request and prepares the HTML for the user authentication page.

[0556] The server returns the HTML as an HTTP response.

[0557] Step 2:

[0558] The user accesses the login screen on the device and enters their username and password.

[0559] Input: Username and Password

[0560] Output: Login credentials

[0561] Specific behavior:

[0562] The user enters their username and password on the login page and clicks the submit button.

[0563] The device sends the login credentials to the server.

[0564] Step 3:

[0565] The server receives the entered information, compares it with the database, and performs authentication. It generates an authentication result and sends it to the terminal.

[0566] Input: Login credentials

[0567] Output: Authentication result

[0568] Specific behavior:

[0569] The server checks the username and password against a database query.

[0570] Based on the result of the comparison, the success or failure of the authentication is determined, and an authentication result message is generated.

[0571] An authentication result message is sent to the terminal.

[0572] Displaying and manipulating virtual space

[0573] Step 4:

[0574] After the server successfully logs in, it sends the 3D model of the virtual museum and data on the artworks.

[0575] Input: Login success notification

[0576] Output: 3D model data, art work data

[0577] Specific behavior:

[0578] The server retrieves the virtual museum's 3D model data and artwork data from the database.

[0579] The data is packaged and sent to the device.

[0580] Step 5:

[0581] The data received by the terminal is processed using a rendering engine to display the virtual space.

[0582] Input: 3D model data, art work data

[0583] Output: Rendered image of the virtual space

[0584] Specific behavior:

[0585] The device reads the data it receives and inputs it into a rendering engine such as Unity or Unreal Engine.

[0586] The rendering engine draws the 3D model and displays it on the device's display.

[0587] Step 6:

[0588] Users use a mouse, keyboard, or VR controller to move around the virtual space and view the exhibits.

[0589] Input: User Action

[0590] Output: Movement within the virtual space and change of viewpoint of exhibits

[0591] Specific behavior:

[0592] The user operates an input device (mouse, keyboard, VR controller).

[0593] The device captures input events and sends instructions to the rendering engine.

[0594] The rendering engine updates the screen and moves the user's viewpoint and position.

[0595] Interactive Tours and Sentiment Analysis

[0596] Step 7:

[0597] Users stop in front of a particular piece of art and type their question or comment into a text box.

[0598] Input: User text input

[0599] Output: Input text data

[0600] Specific behavior:

[0601] The user types a question into the text box and clicks the submit button.

[0602] The terminal transmits the input text data to the server.

[0603] Step 8:

[0604] The text data received by the server is analyzed using a natural language processing engine, and the emotion engine is used to infer the user's intentions and emotions.

[0605] Input: Input text data

[0606] Output: analysis results, emotion estimation results

[0607] Specific behavior:

[0608] The server inputs the input text into a natural language processing engine (GPT-4).

[0609] A natural language processing engine analyzes the text, and an emotion engine infers emotions.

[0610] Generate analysis results and emotion estimation results.

[0611] Step 9:

[0612] The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[0613] Input: Analysis results, emotion estimation results

[0614] Output: Customized response data

[0615] Specific behavior:

[0616] The server creates a response message based on the analysis results.

[0617] The response message is customized with emotional information to generate final response data.

[0618] Step 10:

[0619] The response data generated by the server is sent to the terminal and displayed to the user.

[0620] Input: Customized response data

[0621] Output: Display of response message

[0622] Specific behavior:

[0623] The server sends the customized response data to the terminal.

[0624] The terminal receives the response data and displays it to the user.

[0625] Hosting online events and workshops

[0626] Step 11:

[0627] The server notifies registered users of new online events and workshop schedules.

[0628] Input: Event Schedule

[0629] Output: Notification message

[0630] Specific behavior:

[0631] The server generates a new event schedule.

[0632] Notify registered users of event schedules.

[0633] Step 12:

[0634] The device displays the received notification to the user, who then checks the event information.

[0635] Input: Notification message

[0636] Output: Display of event information

[0637] Specific behavior:

[0638] The terminal receives the notification message.

[0639] The device displays the event information to the user.

[0640] Step 13:

[0641] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[0642] Input: User participation application

[0643] Output: Application request

[0644] Specific behavior:

[0645] The user clicks the sign up button.

[0646] The terminal transmits the participation application data to the server.

[0647] Step 14:

[0648] The server accepts the participation application and registers it in the participant list.

[0649] Input: Application request

[0650] Output: Update participant list

[0651] Specific behavior:

[0652] The server receives the subscription request.

[0653] Add the user to the participant list and save the data.

[0654] Step 15:

[0655] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[0656] Input: Participant list

[0657] Output: Virtual room access link

[0658] Specific behavior:

[0659] The server generates a virtual room at the start of the event.

[0660] Send participants the virtual room access link.

[0661] Step 16:

[0662] Users use their devices to access virtual rooms and participate in online events and workshops.

[0663] Input: Virtual room access link

[0664] Output: Joining a virtual room

[0665] Specific behavior:

[0666] The user clicks on the access link and enters the virtual room.

[0667] The device displays the virtual room and helps you participate in the event.

[0668] Step 17:

[0669] Users interact with other participants through their avatars, chatting and exchanging opinions.

[0670] Input: User interaction intent

[0671] Output: Chat messages, exchange of opinions

[0672] Specific behavior:

[0673] Users control avatars and interact with other participants.

[0674] The device sends chat messages and exchanges of opinions to the other party in real time.

[0675] Prompt Sentence Examples

[0676] "When a user stands in front of a particular painting and asks, 'What is the background to this painting?', provide context for the artwork based on the user's emotions."

[0677] (Application example 2)

[0678] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0679] Conventional virtual community systems and online shopping systems are insufficient in providing personalized experiences based on user emotions, making it difficult to improve user satisfaction. Furthermore, it is difficult to provide product suggestions and customized information based on real-time emotion analysis, making it difficult to stimulate users' purchasing motivation.

[0680] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for building a community using avatars in a virtual space and providing online events and workshops, a means for analyzing the content of users' natural language utterances and estimating the users' emotions, a means for an AI curator to immediately respond to users' questions and provide product information, a means for updating display content based on the users' movements and actions in the virtual store, and a means for providing personalized product suggestions and explanations based on the users' emotions. This makes it possible to provide users with personalized experiences and product suggestions in real time.

[0681] A "virtual space" is a digital environment distinct from the real world, in which users can engage in activities and interactions through avatars.

[0682] An "avatar" is a character or icon that represents a user in a virtual space and acts in place of the user's actions and communication.

[0683] A "community" is a group formed by users with common interests and goals, and is a place for interaction and information sharing within a virtual space.

[0684] An "online event" is an event held in a virtual space via the Internet, in which many users can participate simultaneously.

[0685] A "workshop" is a session for learning knowledge or skills on a specific topic, and is conducted in a virtual space.

[0686] "The user's natural language utterances" refer to the text data or voice data entered or spoken by the user, which is what the system analyzes.

[0687] "Means for estimating emotions" refers to technical means for determining a user's emotional state from their natural language and facial expressions, and includes emotion analysis engines, etc.

[0688] An "AI curator" is a virtual entity within the system that uses artificial intelligence to generate appropriate responses to user questions and provide information.

[0689] "Product information" refers to detailed descriptions, usage instructions, comparison information, etc., of the products and services available in the virtual store.

[0690] A "virtual store" is a virtual store available via the Internet where users can browse, select, and purchase products.

[0691] "Movements and actions" refer to the various actions and operations that users perform within the virtual space, and the system reacts based on these actions and updates the displayed content.

[0692] "Means for updating display content" refers to technical means for dynamically changing the information displayed in the virtual space in response to the user's movements and actions.

[0693] "Personalized product suggestions" is a system function that individually recommends appropriate products and services based on the user's emotions and past behavioral data.

[0694] A "natural language processing engine" is a program that analyzes text entered by a user and understands its content and intent.

[0695] A "means for generating a response" is a technical means that has the function of generating appropriate answers or information based on the user's emotions and the content of the question.

[0696] "Detailed Information" includes detailed explanations and background information about specific products or content, and is provided upon request by the user.

[0697] The present invention relates to a virtual store system that builds a community using avatars in a virtual space and has the ability to recognize user emotions. This system offers online events and workshops, analyzes user comments, provides instant responses from AI curators, updates display content based on user movements and actions, and provides personalized product suggestions and explanations. The following describes the specific configuration and functions required to implement the present invention.

[0698] First, the basic configuration of the system involves linking a server and a terminal. The server is responsible for data management and processing, while the terminal functions as the user interface. This allows users to access the virtual space using their own terminal and use various services.

[0699] The server includes an "emotion analysis engine" that analyzes users' natural language utterances to estimate their emotions, and an "AI curator" that responds instantly to users' questions. The emotion analysis engine collects users' text and voice data and identifies their emotions using machine learning algorithms. This analysis uses a deep learning model trained using, for example, Keras.

[0700] Next, when the user logs in, the device receives the 3D model of the virtual store and product data from the server and displays the virtual space using a rendering engine. The user can move around the virtual space using a mouse, keyboard, or VR controller to view and select displayed products.

[0701] As a concrete example, consider the case where a user stops in front of a specific product. When the user enters a question about the product, the device sends the text data to the server, where the intent and sentiment of the question are analyzed using a sentiment analysis engine and natural language processing engine. Based on the analysis results, the AI ​​curator generates appropriately customized detailed information about the product and responds to the user. For example, if a user asks, "What is this product?" the system might respond, "This product is the latest model that was recently released. It features a particularly high-performance battery and is popular with many users."

[0702] For online events and workshops, the server notifies registered users of new event information and accepts participation registrations. When the event begins, a virtual room is created and users can participate in real time. At this time, the conversation data between users is also analyzed by the emotion analysis engine, and the system provides interactive content according to their emotions.

[0703] In this way, the system of the present invention can recognize the user's emotions in a virtual store and provide a more advanced and personalized shopping experience.

[0704] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0705] Step 1:

[0706] The user accesses the login screen on the terminal and enters the username and password. The input data (username and password) is sent to the server. The server compares the received data with the user database, determines whether the authentication was successful or not, and returns the result to the terminal. The terminal displays the login result to the user.

[0707] Step 2:

[0708] The server sends a 3D model of the virtual store and product data to the user who has been successfully authenticated. The device processes the received data using a rendering engine (e.g., Unity) and displays it as a virtual space on the user's display. The user can move around the virtual space using a mouse, keyboard, or VR controller and view the products on display.

[0709] Step 3:

[0710] The user stops in front of a specific product and enters a question or comment into a text box. The device sends the entered text data to the server. The server then analyzes the received text data using a natural language processing engine (e.g., Transformers' BERT model) to understand the content and intent of the input text. It also uses a sentiment analysis engine (e.g., a deep learning model using Keras) to determine the user's emotional state.

[0711] Step 4:

[0712] Based on the analysis results, the server generates appropriate response data from the AI ​​curator that corresponds to the user's sentiment and question. This response data includes detailed product information and personalized suggestions. The server then sends the generated response data to the device, which then displays it to the user.

[0713] Step 5:

[0714] If the user continues to move around the virtual space and shows interest in other products or areas, the server receives movement and action data and dynamically updates the content displayed in the virtual space, adjusting the display to show products and related information that the user is likely to be interested in next.

[0715] Step 6:

[0716] The server periodically notifies users of the schedule of online events and workshops. When a user wishes to register for participation, they send a request to the server from their device. The server accepts the registration, creates a virtual room, and sends an access link to participants when the event starts. Users access the virtual room and participate in the event while interacting with other participants.

[0717] Step 7:

[0718] During online events and workshops, the server analyzes users' comments and behavioral data in real time and provides interactive content that responds to changes in their emotions. For example, if a user shows interest in a particular topic, additional details or related content will be displayed.

[0719] In this way, this system allows the server and terminals to work together, analyzing user emotions in real time and providing personalized shopping experiences and online event experiences.

[0720] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0721] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0722] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0723] [Second embodiment]

[0724] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0725] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0726] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0727] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0728] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0729] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0730] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0731] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0732] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0733] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0734] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0735] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0736] This invention is a system for providing an art appreciation experience in a virtual space. Specifically, it builds an avatar community and holds online events and workshops. It also includes a function to analyze the user's natural language comments and estimate their emotions. Furthermore, it has a function that allows an AI curator to instantly respond to the user's questions and provide information about the artworks.

[0737] System overview and basic processing

[0738] The virtual museum system works in cooperation with a server and terminals. The server is responsible for data management and processing, while the terminals function as the user interface.

[0739] User authentication and login

[0740] 1. The server accepts the system startup and provides a page for user authentication.

[0741] 2. The user accesses the login screen on the device and enters their username and password.

[0742] 3. The server receives the entered information and authenticates it by checking it against a database.

[0743] 4. The server sends the authentication result to the terminal and notifies it of the success or failure of the login.

[0744] Displaying and manipulating virtual space

[0745] 1. The server sends the virtual museum data to the terminal.

[0746] 2. The device renders the received data and displays the virtual space to the user.

[0747] 3. Users move around the virtual space and view the exhibits.

[0748] Interactive Tours and Sentiment Analysis

[0749] 1. The user enters questions or comments about the artwork in text on the device.

[0750] 2. The server analyzes the content of the speech using a natural language processing algorithm.

[0751] 3. The server infers the user's emotions and intentions and generates an appropriate response.

[0752] 4. The server sends the response to the terminal and displays it to the user.

[0753] Hosting online events and workshops

[0754] 1. The server manages and notifies schedules for online events and workshops.

[0755] 2. The user checks the event information on the device and applies to participate.

[0756] 3. The server manages the participant list and controls participant access when the event starts.

[0757] 4. Users can join the event on their devices and interact with other participants through their avatars.

[0758] Specific examples

[0759] Example 1: User login and virtual space display

[0760] Scenario 1: User A logs in to the virtual museum

[0761] 1. User A accesses the login screen on the device and enters their username and password.

[0762] 2. The server receives User A's authentication information and checks it against the user data in its database.

[0763] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[0764] 4. The device renders the received data and displays the virtual space to User A.

[0765] Example 2: User questions and AI curator responses

[0766] Scenario 2: User B asks about a specific painting

[0767] 1. User B moves through the virtual space and stops in front of a specific painting.

[0768] 2. User B types "What is the background of this picture?" into the text box.

[0769] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms.

[0770] 4. Based on the analysis results, the server generates background information for the painting and returns a response to User B.

[0771] 5. User B reviews the response and learns more about the painting.

[0772] Example 3: Participating in an online workshop

[0773] Scenario 3: User C attends an online workshop

[0774] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[0775] 2. User C checks the event information on their device and applies to participate.

[0776] 3. At the start of the event, the server creates a virtual room and invites participants.

[0777] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[0778] From the above explanation, it can be understood that the present invention provides a virtual museum experience that allows users to easily enjoy the charm of art even in today's busy world. This allows users to enjoy a deep art experience without being restricted by physical constraints.

[0779] The processing flow will be explained below.

[0780] User login process

[0781] Step 1:

[0782] The user operates the device and accesses the login screen, enters their username and password, and clicks the "Login" button.

[0783] Step 2:

[0784] The terminal generates a request to send the entered user name and password to the server, and sends it to the server.

[0785] Step 3:

[0786] The server checks the received username and password against a database to see if the user is valid.

[0787] Step 4:

[0788] The server generates an authentication result and sends a success or failure message to the terminal.

[0789] Step 5:

[0790] The terminal receives the authentication result and displays a login success message to the user, or an error message if authentication fails.

[0791] Displaying and manipulating virtual space

[0792] Step 1:

[0793] The server prepares to send the virtual museum's 3D models and artwork data to users who have successfully logged in.

[0794] Step 2:

[0795] The server sends the virtual museum data to the terminal.

[0796] Step 3:

[0797] The device processes the virtual museum data received using a rendering engine and displays the virtual space on the user's display.

[0798] Step 4:

[0799] Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[0800] Step 5:

[0801] The device detects the user's actions and reflects the corresponding actions (movement, change of viewpoint, selection of specific objects) in the virtual space in real time.

[0802] Interactive Tours and Sentiment Analysis

[0803] Step 1:

[0804] Users stop in front of a particular piece of art and type their question or comment into a text box.

[0805] Step 2:

[0806] The terminal generates a request for transmitting the input text data to the server, and transmits the request to the server.

[0807] Step 3:

[0808] The text data received by the server is analyzed using a natural language processing engine to infer the user's intentions and emotions.

[0809] Step 4:

[0810] The server generates appropriate response data based on the analysis results, including background information about the work, information about the creator, historical context, etc.

[0811] Step 5:

[0812] The server sends the generated response data to the terminal.

[0813] Step 6:

[0814] The terminal displays the received response data to the user, who then checks the detailed information.

[0815] Hosting online events and workshops

[0816] Step 1:

[0817] The server generates data to notify registered users of new online event and workshop schedules.

[0818] Step 2:

[0819] The server sends the notification data to the registered user's terminal.

[0820] Step 3:

[0821] The device displays the received notification to the user, who then checks the event information.

[0822] Step 4:

[0823] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[0824] Step 5:

[0825] The server accepts the participation application and registers it in the participant list.

[0826] Step 6:

[0827] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[0828] Step 7:

[0829] Users use their devices to access virtual rooms and participate in online events and workshops.

[0830] Step 8:

[0831] Users interact with other participants through their avatars, chatting and exchanging opinions.

[0832] Through the above processing steps, the virtual museum system provides users with a realistic art experience and the ability to acquire knowledge.

[0833] Example 1

[0834] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0835] Currently, many people find it difficult to visit museums and galleries due to physical constraints. However, online art appreciation experiences have limitations, and traditional methods do not allow users to enjoy a deep art experience or interact with other viewers. Furthermore, they lack the ability to provide relevant information about artworks and interactive responses based on the user's emotions. Given these factors, a new system is needed to provide a realistic art appreciation experience in a virtual space.

[0836] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0837] In this invention, the server includes means for transmitting a username and password entered from a terminal to the server for user authentication, comparing the username and password with a database and returning the authentication result, means for transmitting data of the virtual space from the server to the terminal and rendering and displaying the received data by the terminal, and means for the server to analyze the text data using a natural language processing engine when a user stops in front of a specific artwork in the virtual space and enters a question or comment, and generate and return an appropriate response. This allows users to enjoy a deep art experience and interact with other viewers without being restricted by physical constraints.

[0838] A "virtual space" is an imaginary space generated on a computer system that users can explore and manipulate as a 3D model.

[0839] An "avatar" is a digital character that users use to represent themselves in a virtual space.

[0840] An "online event" is a virtual event or gathering that takes place in real time over the Internet, allowing participants to share a common experience in a virtual space.

[0841] A "workshop" is an interactive learning or activity space based on a specific theme, where users can learn as participants by interacting with instructors and other participants.

[0842] "Natural language processing" is a technology that allows computers to analyze and understand human language (natural language), and is used to enable machines to understand and appropriately process user statements and input.

[0843] "Sentiment analysis" is a technique for inferring a user's emotional state from their speech or text, and is used to understand the user's current feelings and intentions.

[0844] "AI Curator" is a system based on artificial intelligence technology that acts as a virtual guide, providing instant information in response to user questions.

[0845] "User authentication" is the process of verifying the identity of a user by checking the credentials (username and password) entered by the user to access a system.

[0846] "Rendering" is the process by which a computer visually displays an image or video based on received 3D model data or other graphics data.

[0847] A "natural language processing engine" is a collection of algorithms and software that analyzes a user's text data, understands its content, and responds appropriately.

[0848] This invention is a system for providing an art appreciation experience in a virtual space. Specifically, it builds an avatar community and holds online events and workshops. It also includes a function to analyze the user's natural language comments and estimate their emotions. Furthermore, it has a function that allows an AI curator to instantly respond to the user's questions and provide information about the artworks.

[0849] System Configuration

[0850] This system operates in cooperation with a server and a terminal. The server is responsible for data management and processing, and the terminal functions as a user interface. The main components of this system and their functions are explained below.

[0851] User authentication and login

[0852] The server accepts system startup and provides a user authentication page. The user accesses the login screen on their terminal and enters their username and password. The server receives the entered information and performs authentication by checking it against a database (for example, MySQL or PostgreSQL). It sends the authentication result to the terminal and notifies them of the success or failure of the login. This process uses the web server Apache or Nginx.

[0853] Displaying and manipulating virtual space

[0854] The server sends the virtual museum data to the device, which then renders it using libraries such as WebGL and Three.js to display the virtual space to the user. The user can then move around the virtual space using a keyboard, mouse, or touchscreen to view the exhibits.

[0855] Interactive Tours and Sentiment Analysis

[0856] When a user enters a question or comment about an artwork into the device, the server analyzes the text data using a natural language processing algorithm (e.g., Google NLP API, SpaCy, or NLTK). The server estimates the user's emotions and intentions and generates an appropriate response. The response can be generated using a generative AI model (e.g., OpenAI GPT-3). The server then sends the generated response to the device and displays it to the user.

[0857] Hosting online events and workshops

[0858] The server manages and notifies schedules for online events and workshops. Users can check event information on their devices and apply to participate. The server manages the participant list and creates a virtual room at the start of the event to invite participants. Users can access the virtual room on their devices and interact with other participants through avatars.

[0859] Specific examples

[0860] The following are specific examples of this system:

[0861] Example 1: User login and virtual space display

[0862] When User A logs in to the virtual museum

[0863] 1. User A accesses the login screen on the device and enters their username and password.

[0864] 2. The server receives User A's authentication information and checks it against the user data in its database.

[0865] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[0866] 4. The device renders the received data and displays the virtual space to User A.

[0867] Example 2: User questions and AI curator responses

[0868] User B asks about a specific painting

[0869] 1. User B moves through the virtual space and stops in front of a specific painting.

[0870] 2. User B types "What is the background of this picture?" into the text box.

[0871] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms.

[0872] 4. The server generates background information for the painting based on the analysis results and returns a response to User B.

[0873] 5. User B reviews the response and learns more about the painting.

[0874] Example 3: Participating in an online workshop

[0875] User C participates in an online workshop

[0876] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[0877] 2. User C checks the event information on their device and applies to participate.

[0878] 3. At the start of the event, the server creates a virtual room and invites participants.

[0879] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[0880] Example prompt sentence:

[0881] "What is the background to this painting?"

[0882] "Tell me about Monet's works."

[0883] "When is the next online event?"

[0884] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0885] Handling user authentication and login

[0886] Step 1:

[0887] The server accepts the system startup and provides a page for user authentication.

[0888] Specific operation: When the server receives a startup signal, it uses a web server (Apache or Nginx) to generate an authentication page and sends the URL to the terminal.

[0889] Input: Start signal

[0890] Output: URL of authentication page

[0891] Step 2:

[0892] The user accesses the login screen on the device and enters their username and password.

[0893] Specific operation: The user uses a browser to access the authentication page displayed on the device, enters their username and password, and clicks the submit button.

[0894] Input: Username, Password

[0895] Output: Sending authentication information

[0896] Step 3:

[0897] The server receives the entered information and authenticates it by checking it against a database.

[0898] Specific operation: The server receives the authentication information sent from the terminal and checks it against the user data in the database (MySQL or PostgreSQL) using an SQL query to confirm that it matches.

[0899] Input: Username, Password

[0900] Output: Authentication result

[0901] Step 4:

[0902] The server sends the authentication result to the terminal, notifying it of the success or failure of the login.

[0903] Specific operation: The server formats the authentication result as an HTTP response and sends it to the terminal. The terminal receives the result and redirects the user to the virtual museum screen if the login is successful, or displays an error message if the login is unsuccessful.

[0904] Input: Authentication result

[0905] Output: Login success message or error message

[0906] Virtual space display and operation processing

[0907] Step 1:

[0908] The server sends the virtual museum data to the terminal.

[0909] Specific operation: The server generates the 3D model data (JSON or XML format) required for the virtual museum and sends it to the terminal.

[0910] Input: Request after successful login

[0911] Output: 3D model data

[0912] Step 2:

[0913] The device renders the received data and displays the virtual space to the user.

[0914] Specific operation: The device renders the received 3D model data using graphics libraries such as WebGL or Three.js, and displays the virtual space on the browser.

[0915] Input: 3D model data

[0916] Output: Display of virtual space

[0917] Step 3:

[0918] Users move around the virtual space and view the exhibits.

[0919] Specific actions: Users navigate the virtual space using a keyboard, mouse, or touchscreen, and can click on specific exhibits to view more information.

[0920] Input: User actions

[0921] Output:Detailed information about the exhibit

[0922] Interactive tour and sentiment analysis processing

[0923] Step 1:

[0924] Users enter questions and comments about the artwork via text on their device.

[0925] Specific behavior: The user enters a question or comment in the text box and clicks the submit button.

[0926] Input: Text data (questions and comments)

[0927] Output: Sending text data

[0928] Step 2:

[0929] The server analyzes the content of the speech using natural language processing algorithms.

[0930] Specific operation: The server receives the text data, analyzes it using a natural language processing engine (Google NLP API, SpaCy, NLTK), and infers the user's emotions and intentions.

[0931] Input: Text data

[0932] Output: Analysis results (emotions and intentions)

[0933] Step 3:

[0934] The server estimates the user's emotions and intentions and generates an appropriate response.

[0935] Specific operation: The server generates an appropriate response using a generative AI model (OpenAI GPT-3) based on the analysis results.

[0936] Input: Analysis results

[0937] Output: Response text

[0938] Step 4:

[0939] The server sends the response to the terminal and displays it to the user.

[0940] Specific operation: The server formats the generated response text as an HTTP response and sends it to the terminal, which then displays the response text to the user.

[0941] Input: Response text

[0942] Output: View the response

[0943] Handling online events and workshops

[0944] Step 1:

[0945] The server manages and notifies schedules for online events and workshops.

[0946] Specific operation: The server stores the schedule of the online event in a database and notifies participating users via email or other means.

[0947] Input: Online event information

[0948] Output: Notification message

[0949] Step 2:

[0950] The user checks the event information on their device and applies to participate.

[0951] Specific operation: The user checks the event information on the device and presses the participation application button.

[0952] Input: Event participation application

[0953] Output: Send application information

[0954] Step 3:

[0955] The server manages the participant list and controls participant access at the start of the event.

[0956] Specific operation: The server manages the participant list, creates a virtual room at the start of the event, and invites participants.

[0957] Input: Participant list

[0958] Output: Virtual room URL

[0959] Step 4:

[0960] Users join events on their devices and interact with other participants through avatars.

[0961] Specific operation: A user accesses the URL of a virtual room and interacts with other participants through their avatars.

[0962] Input: Virtual room URL

[0963] Output: Interaction through avatars

[0964] (Application example 1)

[0965] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0966] Modern retail experiences require users to explore a wide variety of products without physical constraints and to receive real-time information to support purchasing decisions. However, existing online shopping systems require a lot of effort for users to obtain detailed product information and receive appropriate recommendations, and they face challenges in providing personalized responses based on the user's emotions and intent. Furthermore, there is a lack of an environment for effectively hosting new product launches and fashion events online.

[0967] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0968] In this invention, the server includes: a means for building a community using avatars in a virtual space and providing online events and workshops; a means for analyzing users' natural language utterances and estimating their emotions; a means for an AI curator to instantly respond to users' questions and provide information; a means for updating display content based on the user's movements and actions in the virtual store; a means for searching for items in the virtual store in natural language using a smartphone or head-mounted display and displaying the results; a means for making real-time suggestions for recommended items and outfits; and a means for managing and notifying the schedule of online fashion events and workshops. This allows users to freely move around the virtual store using their own avatars, efficiently explore a variety of products, and receive personalized information and recommendations in real time. Furthermore, it also becomes possible to effectively hold new product launches and fashion events online, significantly improving the user experience.

[0969] "Virtual space" refers to a digital environment in which users can operate in a virtual realm that is different from their actual physical environment.

[0970] An "avatar" is a digital character with which a user interacts in a virtual space.

[0971] A "community" is a group of users who share a common purpose or interest and interact with each other.

[0972] An "online event" is a real-time event or occasion held over the Internet.

[0973] A "workshop" is a participatory educational and training setting for learning specific skills and knowledge.

[0974] "Natural language" refers to the language used by humans on a daily basis, which needs to be converted into a form that is easy for computers to understand.

[0975] "Emotion" is a concept that represents a human mental state, such as joy, sadness, or anger.

[0976] "AI Curator" refers to a system that uses artificial intelligence to respond to user questions and provide information.

[0977] "Document" does not necessarily mean text data, but may also include drawings, explanatory text, diagrams, etc.

[0978] "Display content" is the collection of information that a user can visually see on the screen.

[0979] A "smartphone" is a portable information terminal that can use a variety of applications in addition to the functions of a mobile phone.

[0980] A "head-mounted display" is a display device worn on the head, and is primarily used to experience virtual reality and augmented reality.

[0981] "Items" are goods or merchandise that users can view or purchase within the virtual space.

[0982] "Recommendation" refers to the presentation of specific products or information based on the user's preferences and behavior.

[0983] "Coordination suggestions" refers to providing recommendations on how to effectively combine fashion items.

[0984] "Managing your schedule" refers to scheduling and coordinating specific events and tasks.

[0985] "Notification" refers to the sending of a message to a user informing them of important information or the onset of an event.

[0986] This invention is a system that allows users to use avatars in a virtual space to explore products and engage in two-way communication, such as asking questions and sending comments. Specifically, using devices such as smartphones or head-mounted displays, users can access a virtual store, search for and view items in real time, and obtain information about artworks.

[0987] System overview and basic processing

[0988] The virtual fashion boutique system includes a smartphone, a head-mounted display, and a server as its main hardware and software components. Specific software components include a natural language processing engine and a sentiment analysis engine.

[0989] User authentication and login

[0990] The server accepts system startup and provides a page for user authentication. Using a terminal, the user accesses the login screen and enters their username and password. The server receives the entered information and performs authentication by comparing it with a database. If authentication is successful, the server sends the virtual store data to the terminal and notifies it that login was successful.

[0991] Displaying and manipulating virtual space

[0992] The server sends the virtual boutique data to the device, which then renders the received data and displays the virtual space to the user, allowing the user to move around the virtual space and view the exhibited products.

[0993] Natural Language Search and Sentiment Analysis

[0994] Users input questions and opinions about fashion items via text. The device sends the input data to the server, which then analyzes the text using a natural language processing engine. The server then infers the user's emotions and intentions, generates an appropriate response, and sends it to the device.

[0995] Display of product information and AI curator response

[0996] When a user stops in front of a specific item, the server generates detailed information related to that item and displays it to the user. The AI ​​curator instantly responds to the user's questions and provides relevant product information.

[0997] Hosting online events and workshops

[0998] The server schedules new product launches and fashion events and notifies users. Users use their devices to check event information and register for participation. The server manages the participant list and controls participant access when the event starts. The device displays the event in real time and allows users to interact with other participants.

[0999] Specific examples

[1000] Example 1: User login and virtual space display

[1001] Scenario 1: A user logs into a virtual fashion boutique

[1002] The user accesses the login screen on their smartphone and enters their username and password. The server receives the user's authentication information and compares it with the user data in the database. If authentication is successful, the server sends the virtual boutique data to the device along with a "Login successful" message. The device then renders the received data and displays the virtual space to the user.

[1003] Example 2: User questions and AI curator responses

[1004] Scenario 2: User asks about a specific dress

[1005] The user moves through the virtual space and stops in front of a particular dress. The user types "What is the trend for this dress?" into a text box. The device sends the input data to the server, which analyzes the text using a natural language processing engine. Based on the analysis results, the server generates trend information for the dress and returns a response to the user. The user checks the response and deepens their knowledge about the dress.

[1006] Example 3: Participating in an online fashion event

[1007] Scenario 3: User attends an online fashion event

[1008] The server notifies users of the schedule for a new online fashion event, "Introducing the Latest Trends." Users check the event information on their devices and apply to participate. When the event starts, the server creates a virtual room and invites participants. Users access the virtual room and participate in the event while interacting with other participants.

[1009] Prompt Sentence Examples

[1010] A user logs into a fashion boutique and asks, "Tell me about the latest dress trends." An AI curator then answers the question, and the user is invited to a new fashion event.

[1011] With the above-described configuration, the present invention realizes a virtual store where a wide variety of products can be efficiently and effectively searched for without being restricted by physical movement or time.

[1012] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1013] Step 1:

[1014] The server accepts the system startup and provides a page for user authentication.

[1015] Input: User's access request

[1016] Data calculation: Display of user authentication page

[1017] Output:Login screen

[1018] Step 2:

[1019] The user accesses the login screen on the device and enters their username and password.

[1020] Input: Username, Password

[1021] Data Computation: Input Data Collection and Formatting

[1022] Output: Authentication data

[1023] Step 3:

[1024] The server receives the entered information and authenticates it by checking it against a database.

[1025] Input: Authentication data

[1026] Data calculation: database matching, authentication calculation

[1027] Output: Authentication result

[1028] Step 4:

[1029] The server sends the authentication result to the terminal, notifying it of the success or failure of the login.

[1030] Input: Authentication result

[1031] Data Calculation: Message Generation

[1032] Output: Notification of successful or failed login

[1033] Step 5:

[1034] The server sends the virtual boutique data to the terminal.

[1035] Input: Login success notification

[1036] Data calculation: data packaging, transmission processing

[1037] Output: Virtual boutique data

[1038] Step 6:

[1039] The device renders the received data and displays the virtual space to the user.

[1040] Input: Virtual Boutique Data

[1041] Data calculation: data rendering, graphics processing

[1042] Output: Virtual space display

[1043] Step 7:

[1044] Users move around the virtual space and view the exhibits.

[1045] Input: User Action

[1046] Data calculation: Position update, display update

[1047] Output: Updated display

[1048] Step 8:

[1049] The user enters a text question about a particular item.

[1050] Input: User text input

[1051] Data calculation: Collecting and structuring input data

[1052] Output: Question data

[1053] Step 9:

[1054] The terminal sends the input data to the server.

[1055] Input: Question data

[1056] Data calculation: Data transmission processing

[1057] Output: Data sent

[1058] Step 10:

[1059] The server analyzes the text using a natural language processing engine.

[1060] Input: Data sent

[1061] Data Computing: Natural Language Analysis, Text Semantic Understanding

[1062] Output: Analysis results

[1063] Step 11:

[1064] The server estimates the user's emotions and intentions, generates an appropriate response, and sends it to the device.

[1065] Input: Analysis results

[1066] Data calculation: emotion estimation, response generation

[1067] Output: Generated response data

[1068] Step 12:

[1069] When a user stops in front of a particular item, the server generates and displays detailed information related to that item.

[1070] Input: Item stop event

[1071] Data Calculations: Finding and Generating More Information

[1072] Output: Show detailed information

[1073] Step 13:

[1074] The server manages the schedule of new product launches and fashion events and notifies users.

[1075] Input: Event schedule information

[1076] Data calculation: Schedule management, notification generation

[1077] Output: Notification message

[1078] Step 14:

[1079] The user checks the event information on the device and applies to participate.

[1080] Input: Event participation application

[1081] Data processing: collection and transmission of application data

[1082] Output: Participation application data

[1083] Step 15:

[1084] The server manages the participant list and controls participant access at the start of the event.

[1085] Input: Application data

[1086] Data operations: list management, access control

[1087] Output: Participant access control

[1088] Step 16:

[1089] The device displays the event in real time and allows users to interact with other participants.

[1090] Input: Real-time event data

[1091] Data calculation: Event display and AC function provided

[1092] Output: Real-time event display, interaction function

[1093] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1094] This invention relates to a system for providing an art appreciation experience in a virtual space, and in particular to a combined emotion engine that recognizes user emotions. Specifically, it includes functions for building an avatar community and hosting online events and workshops, analyzing users' natural language utterances to estimate their emotions, providing information about artworks through an AI curator that responds to users' questions in real time, and updating the display content based on the user's movements and actions in the virtual museum.

[1095] System overview and basic processing

[1096] The virtual museum system works in conjunction with a server and terminals. The server is responsible for data management and processing, while the terminals function as the user interface. Furthermore, an emotion engine is built in, which analyzes user emotion data and provides feedback to the entire system.

[1097] User authentication and login

[1098] 1. The server accepts the system startup and provides a page for user authentication.

[1099] 2. The user accesses the login screen on the device and enters their username and password.

[1100] 3. The server receives the entered information and authenticates it by checking it against a database.

[1101] 4. The server sends the authentication result to the terminal and notifies it of the success or failure of the login.

[1102] Displaying and manipulating virtual space

[1103] 1. The server sends the virtual museum's 3D model and artwork data to the user who has successfully logged in.

[1104] 2. The device processes the received data using a rendering engine and displays the virtual space on the user's display.

[1105] 3. Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[1106] Interactive Tours and Sentiment Analysis

[1107] 1. Users stop in front of a particular piece of art and type their question or comment into a text box.

[1108] 2. The terminal sends the entered text data to the server.

[1109] 3. The server analyzes the received text data using a natural language processing engine and uses an emotion engine to infer the user's intentions and emotions.

[1110] 4. The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[1111] 5. The server generates a response and sends it to the terminal, where it displays it to the user.

[1112] Hosting online events and workshops

[1113] 1. The server notifies registered users of new online events and workshops scheduled.

[1114] 2. The device displays the received notification to the user, and the user checks the event information.

[1115] 3. If the user wishes to register, he / she sends a participation application request to the server from his / her terminal.

[1116] 4. The server accepts the participation application and registers it on the participant list.

[1117] 5. At the start of the event, the server creates a virtual room for participants and sends them an access link.

[1118] 6. Users use their devices to access the virtual room and participate in online events and workshops.

[1119] 7. Users can interact with other participants through their avatars, chat and exchange opinions.

[1120] Specific examples

[1121] Example 1: User login and sentiment engine analysis

[1122] Scenario 1: User A logs in to the virtual museum

[1123] 1. User A accesses the login screen on the device and enters their username and password.

[1124] 2. The server receives User A's authentication information and checks it against the user data in its database.

[1125] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[1126] 4. The device renders the received data and displays the virtual space to User A.

[1127] Example 2: User Questions and Emotion Engine Responses

[1128] Scenario 2: User B asks about a specific painting

[1129] 1. User B moves through the virtual space and stops in front of a specific painting.

[1130] 2. User B types "What is the background of this picture?" into the text box.

[1131] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms and an emotion engine.

[1132] 4. Based on the analysis results, the server generates customized information based on the painting's background information and the user's emotions, and returns a response to User B.

[1133] 5. User B reviews the response and learns more about the painting.

[1134] Example 3: Online workshops and the use of sentiment analysis

[1135] Scenario 3: User C attends an online workshop

[1136] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[1137] 2. User C checks the event information on their device and applies to participate.

[1138] 3. At the start of the event, the server creates a virtual room and invites participants.

[1139] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[1140] 5. The server analyzes users' comments in real time during the workshop using an emotion engine and provides interactive content that responds to participants' emotional changes.

[1141] As explained above, by combining an emotion engine that recognizes the user's emotions, the present invention can provide a more personalized art experience and deepen the user's appreciation experience, allowing the user to enjoy a deeper art experience without being restricted by physical constraints.

[1142] The processing flow will be explained below.

[1143] User login process

[1144] Step 1:

[1145] The user operates the device and accesses the login screen, enters their username and password, and clicks the "Login" button.

[1146] Step 2:

[1147] The terminal generates a request to send the entered user name and password to the server, and sends it to the server.

[1148] Step 3:

[1149] The server checks the received username and password against a database to see if the user is valid.

[1150] Step 4:

[1151] The server generates an authentication result and sends a success or failure message to the terminal.

[1152] Step 5:

[1153] The terminal receives the authentication result and displays a login success message to the user, or an error message if authentication fails.

[1154] Displaying and manipulating virtual space

[1155] Step 1:

[1156] The server prepares to send the virtual museum's 3D models and artwork data to users who have successfully logged in.

[1157] Step 2:

[1158] The server sends the virtual museum data to the terminal.

[1159] Step 3:

[1160] The device processes the virtual museum data received using a rendering engine and displays the virtual space on the user's display.

[1161] Step 4:

[1162] Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[1163] Step 5:

[1164] The device detects the user's actions and reflects the corresponding actions (movement, change of viewpoint, selection of specific objects) in the virtual space in real time.

[1165] Interactive Tours and Sentiment Analysis

[1166] Step 1:

[1167] Users stop in front of a particular piece of art and type their question or comment into a text box.

[1168] Step 2:

[1169] The terminal generates a request for transmitting the input text data to the server, and transmits the request to the server.

[1170] Step 3:

[1171] The text data received by the server is analyzed using a natural language processing engine, and the emotion engine is used to infer the user's intentions and emotions.

[1172] Step 4:

[1173] The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[1174] Step 5:

[1175] The server sends the generated response data to the terminal.

[1176] Step 6:

[1177] The terminal displays the received response data to the user, who then checks the detailed information.

[1178] Hosting online events and workshops

[1179] Step 1:

[1180] The server generates data to notify registered users of new online event and workshop schedules.

[1181] Step 2:

[1182] The server sends the notification data to the registered user's terminal.

[1183] Step 3:

[1184] The device displays the received notification to the user, who then checks the event information.

[1185] Step 4:

[1186] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[1187] Step 5:

[1188] The server accepts the participation application and registers it in the participant list.

[1189] Step 6:

[1190] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[1191] Step 7:

[1192] Users use their devices to access virtual rooms and participate in online events and workshops.

[1193] Step 8:

[1194] Users interact with other participants through their avatars, chatting and exchanging opinions.

[1195] Step 9:

[1196] During the workshop, the server analyzes users' comments in real time using an emotion engine and provides interactive content that responds to participants' emotional changes.

[1197] Through the above processing steps, the virtual museum system provides users with an immersive art experience and knowledge acquisition, and by combining it with an emotion engine, it can realize a personalized experience based on the user's emotions.

[1198] Example 2

[1199] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1200] Virtual art appreciation lacks a personalized experience that takes into account the user's emotions. This can lead to a decrease in user satisfaction and motivation to participate. Furthermore, while users need fast and accurate responses to their questions, existing systems lack immediacy and adaptability.

[1201] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for building a community using avatars in a virtual space and providing online events and workshops; a means for analyzing the content of users' natural language utterances and estimating the user's emotions; a means for an AI curator to instantly respond to users' questions and provide information about artworks; a means for updating display content based on the user's movements and actions in the virtual museum; a means for performing user authentication and login processing; a means for displaying the virtual space using a user interface rendering engine; a means for analyzing input text data using a natural language processing engine and estimating the user's intentions and emotions using an emotion engine; and a means for generating response data based on the analysis results and displaying it to the user. This enables a personalized art appreciation experience that takes user emotions into consideration, thereby improving user satisfaction and willingness to participate. It also enables rapid and accurate responses, providing a more interactive and engaging experience.

[1202] A "virtual space" is a virtual environment generated using computers and virtual reality technology that users can experience through their senses of sight, hearing, touch, etc.

[1203] An "avatar" is a character or alter ego that a user uses to represent themselves in a virtual space.

[1204] A "community" is a group of users who come together under a common purpose or interest and interact and cooperate with each other.

[1205] An "online event" is a gathering or event that takes place in real time over the internet.

[1206] A "workshop" is an event held for participants to learn about a particular topic or technology and share practical experience.

[1207] "User authentication" means the process of verifying that a user attempting to access a system is a registered and authorized user.

[1208] A "natural language processing engine" is a computer algorithm or program that analyzes, understands, and generates the natural language used by humans on a daily basis.

[1209] An "emotion engine" is an algorithm or program that analyzes and infers a user's emotions from their text or voice data.

[1210] An "artificial intelligence curator" is a system that uses AI technology to provide information in response to user questions and operations, and to explain artworks.

[1211] A "virtual museum" is a museum in a virtual space set up on the Internet, where digital artworks and exhibits can be viewed.

[1212] A "rendering engine" is a program or software that generates computer graphics and displays them on a display.

[1213] A "session" refers to a series of operations and communications between when a user logs in to the system and when they log out, and is a temporary unit of communication that is specific to each user.

[1214] This invention relates to a system for providing an art appreciation experience in a virtual space, particularly incorporating an emotion engine that recognizes the user's emotions. Users can access the virtual space using an avatar to view exhibits and participate in online events and workshops. It also has the ability to analyze the user's comments and generate appropriate responses based on their emotions.

[1215] System configuration

[1216] This system works in conjunction with a server and a device. The server manages and processes data, while the device functions as the user interface. It is equipped with an emotion engine and a natural language processing engine, which analyzes user emotions to provide a personalized experience. Specifically, it includes the following components:

[1217] 1. Server

[1218] Database Management System (DBMS): Manages user information and artwork data. For example, MySQL or PostgreSQL can be used.

[1219] Natural language processing engine: Analyzes user utterances. For example, GPT-4 is used.

[1220] Emotion engine: Estimates user emotions.

[1221] Rendering engine: Processes the 3D models to generate the virtual space. For example, Unity or Unreal Engine is used.

[1222] 2. Terminal

[1223] User Interface (UI): Provides access to and control of virtual spaces, including PCs and VR devices.

[1224] Input devices: keyboard, mouse, VR controller, etc.

[1225] 3. Users

[1226] Avatar: A character or alter ego that a user controls in a virtual space.

[1227] Specific processing details

[1228] User authentication and login

[1229] The server starts the system and provides a user authentication page. The user enters their username and password on the terminal and sends them to the server. The server checks the database for authentication and returns the result to the terminal.

[1230] Displaying and manipulating virtual space

[1231] After a successful login, the server sends data on the virtual museum's 3D model and artworks. The device receives the data and renders it using a rendering engine. The user then uses a mouse or VR controller to move around the virtual space.

[1232] Interactive Tours and Sentiment Analysis

[1233] Users stop in front of a particular artwork and enter their questions or comments in text. The device sends the data to a server, where it is analyzed by a natural language processing engine. The emotion engine infers the user's intentions and emotions, and the server generates an appropriate response and sends it to the device.

[1234] Hosting online events and workshops

[1235] The server notifies users of new event and workshop schedules. Users can check the schedules on their devices and apply to participate. The server registers participants on a list and opens a virtual room at the start of the event. Users can access the room and interact with other participants.

[1236] Specific scenarios

[1237] Scenario 1: User Logged In

[1238] User A accesses the login screen on their device and enters their username and password. The server receives the authentication information and checks it against the database. If authentication is successful, the virtual museum data is sent to the device and the virtual space is displayed to User A.

[1239] Scenario 2: User Questions and Sentiment Analysis

[1240] User B asks a question about a particular painting. He types, "What is the background of this painting?" into his device. The text data is sent to the server, where it is analyzed by the natural language processing engine and emotion engine. The server generates a response based on the analysis results and sends it to the device.

[1241] Scenario 3: Participating in an online workshop

[1242] User C receives the schedule for a new online workshop and applies to participate. The server registers the participants in the list and generates a virtual room at the start of the event. User C accesses the room and participates in the workshop.

[1243] Prompt Sentence Examples

[1244] "When a user stands in front of a particular painting and asks, 'What is the background to this painting?', provide context for the artwork based on the user's emotions."

[1245] Through these functions, the present invention can recognize the user's emotions and provide a personalized art appreciation experience, allowing the user to enjoy a deep art experience without being restricted by physical constraints.

[1246] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1247] The flow of this system's program processing

[1248] User authentication and login

[1249] Step 1:

[1250] The server accepts the system startup, provides a user authentication page, generates a URL for the user to access the login screen, and returns it as an HTTP response.

[1251] Input: User's access request

[1252] Output: User authentication page

[1253] Specific behavior:

[1254] The server receives the HTTP request and prepares the HTML for the user authentication page.

[1255] The server returns the HTML as an HTTP response.

[1256] Step 2:

[1257] The user accesses the login screen on the device and enters their username and password.

[1258] Input: Username and Password

[1259] Output: Login credentials

[1260] Specific behavior:

[1261] The user enters their username and password on the login page and clicks the submit button.

[1262] The device sends the login credentials to the server.

[1263] Step 3:

[1264] The server receives the entered information, compares it with the database, and performs authentication. It generates an authentication result and sends it to the terminal.

[1265] Input: Login credentials

[1266] Output: Authentication result

[1267] Specific behavior:

[1268] The server checks the username and password against a database query.

[1269] Based on the result of the comparison, the success or failure of the authentication is determined, and an authentication result message is generated.

[1270] An authentication result message is sent to the terminal.

[1271] Displaying and manipulating virtual space

[1272] Step 4:

[1273] After the server successfully logs in, it sends the 3D model of the virtual museum and data on the artworks.

[1274] Input: Login success notification

[1275] Output: 3D model data, art work data

[1276] Specific behavior:

[1277] The server retrieves the virtual museum's 3D model data and artwork data from the database.

[1278] The data is packaged and sent to the device.

[1279] Step 5:

[1280] The data received by the terminal is processed using a rendering engine to display the virtual space.

[1281] Input: 3D model data, art work data

[1282] Output: Rendered image of the virtual space

[1283] Specific behavior:

[1284] The device reads the data it receives and inputs it into a rendering engine such as Unity or Unreal Engine.

[1285] The rendering engine draws the 3D model and displays it on the device's display.

[1286] Step 6:

[1287] Users use a mouse, keyboard, or VR controller to move around the virtual space and view the exhibits.

[1288] Input: User Action

[1289] Output: Movement within the virtual space and change of viewpoint of exhibits

[1290] Specific behavior:

[1291] The user operates an input device (mouse, keyboard, VR controller).

[1292] The device captures input events and sends instructions to the rendering engine.

[1293] The rendering engine updates the screen and moves the user's viewpoint and position.

[1294] Interactive Tours and Sentiment Analysis

[1295] Step 7:

[1296] Users stop in front of a particular piece of art and type their question or comment into a text box.

[1297] Input: User text input

[1298] Output: Input text data

[1299] Specific behavior:

[1300] The user types a question into the text box and clicks the submit button.

[1301] The terminal transmits the input text data to the server.

[1302] Step 8:

[1303] The text data received by the server is analyzed using a natural language processing engine, and the emotion engine is used to infer the user's intentions and emotions.

[1304] Input: Input text data

[1305] Output: analysis results, emotion estimation results

[1306] Specific behavior:

[1307] The server inputs the input text into a natural language processing engine (GPT-4).

[1308] A natural language processing engine analyzes the text, and an emotion engine infers emotions.

[1309] Generate analysis results and emotion estimation results.

[1310] Step 9:

[1311] The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[1312] Input: Analysis results, emotion estimation results

[1313] Output: Customized response data

[1314] Specific behavior:

[1315] The server creates a response message based on the analysis results.

[1316] The response message is customized with emotional information to generate final response data.

[1317] Step 10:

[1318] The response data generated by the server is sent to the terminal and displayed to the user.

[1319] Input: Customized response data

[1320] Output: Display of response message

[1321] Specific behavior:

[1322] The server sends the customized response data to the terminal.

[1323] The terminal receives the response data and displays it to the user.

[1324] Hosting online events and workshops

[1325] Step 11:

[1326] The server notifies registered users of new online events and workshop schedules.

[1327] Input: Event Schedule

[1328] Output: Notification message

[1329] Specific behavior:

[1330] The server generates a new event schedule.

[1331] Notify registered users of event schedules.

[1332] Step 12:

[1333] The device displays the received notification to the user, who then checks the event information.

[1334] Input: Notification message

[1335] Output: Display of event information

[1336] Specific behavior:

[1337] The terminal receives the notification message.

[1338] The device displays the event information to the user.

[1339] Step 13:

[1340] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[1341] Input: User participation application

[1342] Output: Application request

[1343] Specific behavior:

[1344] The user clicks the sign up button.

[1345] The terminal transmits the participation application data to the server.

[1346] Step 14:

[1347] The server accepts the participation application and registers it in the participant list.

[1348] Input: Application request

[1349] Output: Update participant list

[1350] Specific behavior:

[1351] The server receives the subscription request.

[1352] Add the user to the participant list and save the data.

[1353] Step 15:

[1354] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[1355] Input: Participant list

[1356] Output: Virtual room access link

[1357] Specific behavior:

[1358] The server generates a virtual room at the start of the event.

[1359] Send participants the virtual room access link.

[1360] Step 16:

[1361] Users use their devices to access virtual rooms and participate in online events and workshops.

[1362] Input: Virtual room access link

[1363] Output: Joining a virtual room

[1364] Specific behavior:

[1365] The user clicks on the access link and enters the virtual room.

[1366] The device displays the virtual room and helps you participate in the event.

[1367] Step 17:

[1368] Users interact with other participants through their avatars, chatting and exchanging opinions.

[1369] Input: User interaction intent

[1370] Output: Chat messages, exchange of opinions

[1371] Specific behavior:

[1372] Users control avatars and interact with other participants.

[1373] The device sends chat messages and exchanges of opinions to the other party in real time.

[1374] Prompt Sentence Examples

[1375] "When a user stands in front of a particular painting and asks, 'What is the background to this painting?', provide context for the artwork based on the user's emotions."

[1376] (Application example 2)

[1377] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1378] Conventional virtual community systems and online shopping systems are insufficient in providing personalized experiences based on user emotions, making it difficult to improve user satisfaction. Furthermore, it is difficult to provide product suggestions and customized information based on real-time emotion analysis, making it difficult to stimulate users' purchasing motivation.

[1379] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for building a community using avatars in a virtual space and providing online events and workshops, a means for analyzing the content of users' natural language utterances and estimating the users' emotions, a means for an AI curator to immediately respond to users' questions and provide product information, a means for updating display content based on the users' movements and actions in the virtual store, and a means for providing personalized product suggestions and explanations based on the users' emotions. This makes it possible to provide users with personalized experiences and product suggestions in real time.

[1380] A "virtual space" is a digital environment distinct from the real world, in which users can engage in activities and interactions through avatars.

[1381] An "avatar" is a character or icon that represents a user in a virtual space and acts in place of the user's actions and communication.

[1382] A "community" is a group formed by users with common interests and goals, and is a place for interaction and information sharing within a virtual space.

[1383] An "online event" is an event held in a virtual space via the Internet, in which many users can participate simultaneously.

[1384] A "workshop" is a session for learning knowledge or skills on a specific topic, and is conducted in a virtual space.

[1385] "The user's natural language utterances" refer to the text data or voice data entered or spoken by the user, which is what the system analyzes.

[1386] "Means for estimating emotions" refers to technical means for determining a user's emotional state from their natural language and facial expressions, and includes emotion analysis engines, etc.

[1387] An "AI curator" is a virtual entity within the system that uses artificial intelligence to generate appropriate responses to user questions and provide information.

[1388] "Product information" refers to detailed descriptions, usage instructions, comparison information, etc., of the products and services available in the virtual store.

[1389] A "virtual store" is a virtual store available via the Internet where users can browse, select, and purchase products.

[1390] "Movements and actions" refer to the various actions and operations that users perform within the virtual space, and the system reacts based on these actions and updates the displayed content.

[1391] "Means for updating display content" refers to technical means for dynamically changing the information displayed in the virtual space in response to the user's movements and actions.

[1392] "Personalized product suggestions" is a system function that individually recommends appropriate products and services based on the user's emotions and past behavioral data.

[1393] A "natural language processing engine" is a program that analyzes text entered by a user and understands its content and intent.

[1394] A "means for generating a response" is a technical means that has the function of generating appropriate answers or information based on the user's emotions and the content of the question.

[1395] "Detailed Information" includes detailed explanations and background information about specific products or content, and is provided upon request by the user.

[1396] The present invention relates to a virtual store system that builds a community using avatars in a virtual space and has the ability to recognize user emotions. This system offers online events and workshops, analyzes user comments, provides instant responses from AI curators, updates display content based on user movements and actions, and provides personalized product suggestions and explanations. The following describes the specific configuration and functions required to implement the present invention.

[1397] First, the basic configuration of the system involves linking a server and a terminal. The server is responsible for data management and processing, while the terminal functions as the user interface. This allows users to access the virtual space using their own terminal and use various services.

[1398] The server includes an "emotion analysis engine" that analyzes users' natural language utterances to estimate their emotions, and an "AI curator" that responds instantly to users' questions. The emotion analysis engine collects users' text and voice data and identifies their emotions using machine learning algorithms. This analysis uses a deep learning model trained using, for example, Keras.

[1399] Next, when the user logs in, the device receives the 3D model of the virtual store and product data from the server and displays the virtual space using a rendering engine. The user can move around the virtual space using a mouse, keyboard, or VR controller to view and select displayed products.

[1400] As a concrete example, consider the case where a user stops in front of a specific product. When the user enters a question about the product, the device sends the text data to the server, where the intent and sentiment of the question are analyzed using a sentiment analysis engine and natural language processing engine. Based on the analysis results, the AI ​​curator generates appropriately customized detailed information about the product and responds to the user. For example, if a user asks, "What is this product?" the system might respond, "This product is the latest model that was recently released. It features a particularly high-performance battery and is popular with many users."

[1401] For online events and workshops, the server notifies registered users of new event information and accepts participation registrations. When the event begins, a virtual room is created and users can participate in real time. At this time, the conversation data between users is also analyzed by the emotion analysis engine, and the system provides interactive content according to their emotions.

[1402] In this way, the system of the present invention can recognize the user's emotions in a virtual store and provide a more advanced and personalized shopping experience.

[1403] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1404] Step 1:

[1405] The user accesses the login screen on the terminal and enters the username and password. The input data (username and password) is sent to the server. The server compares the received data with the user database, determines whether the authentication was successful or not, and returns the result to the terminal. The terminal displays the login result to the user.

[1406] Step 2:

[1407] The server sends a 3D model of the virtual store and product data to the user who has been successfully authenticated. The device processes the received data using a rendering engine (e.g., Unity) and displays it as a virtual space on the user's display. The user can move around the virtual space using a mouse, keyboard, or VR controller and view the products on display.

[1408] Step 3:

[1409] The user stops in front of a specific product and enters a question or comment into a text box. The device sends the entered text data to the server. The server then analyzes the received text data using a natural language processing engine (e.g., Transformers' BERT model) to understand the content and intent of the input text. It also uses a sentiment analysis engine (e.g., a deep learning model using Keras) to determine the user's emotional state.

[1410] Step 4:

[1411] Based on the analysis results, the server generates appropriate response data from the AI ​​curator that corresponds to the user's sentiment and question. This response data includes detailed product information and personalized suggestions. The server then sends the generated response data to the device, which then displays it to the user.

[1412] Step 5:

[1413] If the user continues to move around the virtual space and shows interest in other products or areas, the server receives movement and action data and dynamically updates the content displayed in the virtual space, adjusting the display to show products and related information that the user is likely to be interested in next.

[1414] Step 6:

[1415] The server periodically notifies users of the schedule of online events and workshops. When a user wishes to register for participation, they send a request to the server from their device. The server accepts the registration, creates a virtual room, and sends an access link to participants when the event starts. Users access the virtual room and participate in the event while interacting with other participants.

[1416] Step 7:

[1417] During online events and workshops, the server analyzes users' comments and behavioral data in real time and provides interactive content that responds to changes in their emotions. For example, if a user shows interest in a particular topic, additional details or related content will be displayed.

[1418] In this way, this system allows the server and terminals to work together, analyzing user emotions in real time and providing personalized shopping experiences and online event experiences.

[1419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1420] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1421] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1422] [Third embodiment]

[1423] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1424] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1425] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1426] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1427] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1429] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1430] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1431] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1432] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1433] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1434] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1435] This invention is a system for providing an art appreciation experience in a virtual space. Specifically, it builds an avatar community and holds online events and workshops. It also includes a function to analyze the user's natural language comments and estimate their emotions. Furthermore, it has a function that allows an AI curator to instantly respond to the user's questions and provide information about the artworks.

[1436] System overview and basic processing

[1437] The virtual museum system works in cooperation with a server and terminals. The server is responsible for data management and processing, while the terminals function as the user interface.

[1438] User authentication and login

[1439] 1. The server accepts the system startup and provides a page for user authentication.

[1440] 2. The user accesses the login screen on the device and enters their username and password.

[1441] 3. The server receives the entered information and authenticates it by checking it against a database.

[1442] 4. The server sends the authentication result to the terminal and notifies it of the success or failure of the login.

[1443] Displaying and manipulating virtual space

[1444] 1. The server sends the virtual museum data to the terminal.

[1445] 2. The device renders the received data and displays the virtual space to the user.

[1446] 3. Users move around the virtual space and view the exhibits.

[1447] Interactive Tours and Sentiment Analysis

[1448] 1. The user enters questions or comments about the artwork in text on the device.

[1449] 2. The server analyzes the content of the speech using a natural language processing algorithm.

[1450] 3. The server infers the user's emotions and intentions and generates an appropriate response.

[1451] 4. The server sends the response to the terminal and displays it to the user.

[1452] Hosting online events and workshops

[1453] 1. The server manages and notifies schedules for online events and workshops.

[1454] 2. The user checks the event information on the device and applies to participate.

[1455] 3. The server manages the participant list and controls participant access when the event starts.

[1456] 4. Users can join the event on their devices and interact with other participants through their avatars.

[1457] Specific examples

[1458] Example 1: User login and virtual space display

[1459] Scenario 1: User A logs in to the virtual museum

[1460] 1. User A accesses the login screen on the device and enters their username and password.

[1461] 2. The server receives User A's authentication information and checks it against the user data in its database.

[1462] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[1463] 4. The device renders the received data and displays the virtual space to User A.

[1464] Example 2: User questions and AI curator responses

[1465] Scenario 2: User B asks about a specific painting

[1466] 1. User B moves through the virtual space and stops in front of a specific painting.

[1467] 2. User B types "What is the background of this picture?" into the text box.

[1468] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms.

[1469] 4. Based on the analysis results, the server generates background information for the painting and returns a response to User B.

[1470] 5. User B reviews the response and learns more about the painting.

[1471] Example 3: Participating in an online workshop

[1472] Scenario 3: User C attends an online workshop

[1473] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[1474] 2. User C checks the event information on their device and applies to participate.

[1475] 3. At the start of the event, the server creates a virtual room and invites participants.

[1476] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[1477] From the above explanation, it can be understood that the present invention provides a virtual museum experience that allows users to easily enjoy the charm of art even in today's busy world. This allows users to enjoy a deep art experience without being restricted by physical constraints.

[1478] The processing flow will be explained below.

[1479] User login process

[1480] Step 1:

[1481] The user operates the device and accesses the login screen, enters their username and password, and clicks the "Login" button.

[1482] Step 2:

[1483] The terminal generates a request to send the entered user name and password to the server, and sends it to the server.

[1484] Step 3:

[1485] The server checks the received username and password against a database to see if the user is valid.

[1486] Step 4:

[1487] The server generates an authentication result and sends a success or failure message to the terminal.

[1488] Step 5:

[1489] The terminal receives the authentication result and displays a login success message to the user, or an error message if authentication fails.

[1490] Displaying and manipulating virtual space

[1491] Step 1:

[1492] The server prepares to send the virtual museum's 3D models and artwork data to users who have successfully logged in.

[1493] Step 2:

[1494] The server sends the virtual museum data to the terminal.

[1495] Step 3:

[1496] The device processes the virtual museum data received using a rendering engine and displays the virtual space on the user's display.

[1497] Step 4:

[1498] Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[1499] Step 5:

[1500] The device detects the user's actions and reflects the corresponding actions (movement, change of viewpoint, selection of specific objects) in the virtual space in real time.

[1501] Interactive Tours and Sentiment Analysis

[1502] Step 1:

[1503] Users stop in front of a particular piece of art and type their question or comment into a text box.

[1504] Step 2:

[1505] The terminal generates a request for transmitting the input text data to the server, and transmits the request to the server.

[1506] Step 3:

[1507] The text data received by the server is analyzed using a natural language processing engine to infer the user's intentions and emotions.

[1508] Step 4:

[1509] The server generates appropriate response data based on the analysis results, including background information about the work, information about the creator, historical context, etc.

[1510] Step 5:

[1511] The server sends the generated response data to the terminal.

[1512] Step 6:

[1513] The terminal displays the received response data to the user, who then checks the detailed information.

[1514] Hosting online events and workshops

[1515] Step 1:

[1516] The server generates data to notify registered users of new online event and workshop schedules.

[1517] Step 2:

[1518] The server sends the notification data to the registered user's terminal.

[1519] Step 3:

[1520] The device displays the received notification to the user, who then checks the event information.

[1521] Step 4:

[1522] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[1523] Step 5:

[1524] The server accepts the participation application and registers it in the participant list.

[1525] Step 6:

[1526] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[1527] Step 7:

[1528] Users use their devices to access virtual rooms and participate in online events and workshops.

[1529] Step 8:

[1530] Users interact with other participants through their avatars, chatting and exchanging opinions.

[1531] Through the above processing steps, the virtual museum system provides users with a realistic art experience and the ability to acquire knowledge.

[1532] Example 1

[1533] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1534] Currently, many people find it difficult to visit museums and galleries due to physical constraints. However, online art appreciation experiences have limitations, and traditional methods do not allow users to enjoy a deep art experience or interact with other viewers. Furthermore, they lack the ability to provide relevant information about artworks and interactive responses based on the user's emotions. Given these factors, a new system is needed to provide a realistic art appreciation experience in a virtual space.

[1535] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1536] In this invention, the server includes means for transmitting a username and password entered from a terminal to the server for user authentication, comparing the username and password with a database and returning the authentication result, means for transmitting data of the virtual space from the server to the terminal and rendering and displaying the received data by the terminal, and means for the server to analyze the text data using a natural language processing engine when a user stops in front of a specific artwork in the virtual space and enters a question or comment, and generate and return an appropriate response. This allows users to enjoy a deep art experience and interact with other viewers without being restricted by physical constraints.

[1537] A "virtual space" is an imaginary space generated on a computer system that users can explore and manipulate as a 3D model.

[1538] An "avatar" is a digital character that users use to represent themselves in a virtual space.

[1539] An "online event" is a virtual event or gathering that takes place in real time over the Internet, allowing participants to share a common experience in a virtual space.

[1540] A "workshop" is an interactive learning or activity space based on a specific theme, where users can learn as participants by interacting with instructors and other participants.

[1541] "Natural language processing" is a technology that allows computers to analyze and understand human language (natural language), and is used to enable machines to understand and appropriately process user statements and input.

[1542] "Sentiment analysis" is a technique for inferring a user's emotional state from their speech or text, and is used to understand the user's current feelings and intentions.

[1543] "AI Curator" is a system based on artificial intelligence technology that acts as a virtual guide, providing instant information in response to user questions.

[1544] "User authentication" is the process of verifying the identity of a user by checking the credentials (username and password) entered by the user to access a system.

[1545] "Rendering" is the process by which a computer visually displays an image or video based on received 3D model data or other graphics data.

[1546] A "natural language processing engine" is a collection of algorithms and software that analyzes a user's text data, understands its content, and responds appropriately.

[1547] This invention is a system for providing an art appreciation experience in a virtual space. Specifically, it builds an avatar community and holds online events and workshops. It also includes a function to analyze the user's natural language comments and estimate their emotions. Furthermore, it has a function that allows an AI curator to instantly respond to the user's questions and provide information about the artworks.

[1548] System Configuration

[1549] This system operates in cooperation with a server and a terminal. The server is responsible for data management and processing, and the terminal functions as a user interface. The main components of this system and their functions are explained below.

[1550] User authentication and login

[1551] The server accepts system startup and provides a user authentication page. The user accesses the login screen on their terminal and enters their username and password. The server receives the entered information and performs authentication by checking it against a database (for example, MySQL or PostgreSQL). It sends the authentication result to the terminal and notifies them of the success or failure of the login. This process uses the web server Apache or Nginx.

[1552] Displaying and manipulating virtual space

[1553] The server sends the virtual museum data to the device, which then renders it using libraries such as WebGL and Three.js to display the virtual space to the user. The user can then move around the virtual space using a keyboard, mouse, or touchscreen to view the exhibits.

[1554] Interactive Tours and Sentiment Analysis

[1555] When a user enters a question or comment about an artwork into the device, the server analyzes the text data using a natural language processing algorithm (e.g., Google NLP API, SpaCy, or NLTK). The server estimates the user's emotions and intentions and generates an appropriate response. The response can be generated using a generative AI model (e.g., OpenAI GPT-3). The server then sends the generated response to the device and displays it to the user.

[1556] Hosting online events and workshops

[1557] The server manages and notifies schedules for online events and workshops. Users can check event information on their devices and apply to participate. The server manages the participant list and creates a virtual room at the start of the event to invite participants. Users can access the virtual room on their devices and interact with other participants through avatars.

[1558] Specific examples

[1559] The following are specific examples of this system:

[1560] Example 1: User login and virtual space display

[1561] When User A logs in to the virtual museum

[1562] 1. User A accesses the login screen on the device and enters their username and password.

[1563] 2. The server receives User A's authentication information and checks it against the user data in its database.

[1564] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[1565] 4. The device renders the received data and displays the virtual space to User A.

[1566] Example 2: User questions and AI curator responses

[1567] User B asks about a specific painting

[1568] 1. User B moves through the virtual space and stops in front of a specific painting.

[1569] 2. User B types "What is the background of this picture?" into the text box.

[1570] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms.

[1571] 4. The server generates background information for the painting based on the analysis results and returns a response to User B.

[1572] 5. User B reviews the response and learns more about the painting.

[1573] Example 3: Participating in an online workshop

[1574] User C participates in an online workshop

[1575] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[1576] 2. User C checks the event information on their device and applies to participate.

[1577] 3. At the start of the event, the server creates a virtual room and invites participants.

[1578] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[1579] Example prompt sentence:

[1580] "What is the background to this painting?"

[1581] "Tell me about Monet's works."

[1582] "When is the next online event?"

[1583] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1584] Handling user authentication and login

[1585] Step 1:

[1586] The server accepts the system startup and provides a page for user authentication.

[1587] Specific operation: When the server receives a startup signal, it uses a web server (Apache or Nginx) to generate an authentication page and sends the URL to the terminal.

[1588] Input: Start signal

[1589] Output: URL of authentication page

[1590] Step 2:

[1591] The user accesses the login screen on the device and enters their username and password.

[1592] Specific operation: The user uses a browser to access the authentication page displayed on the device, enters their username and password, and clicks the submit button.

[1593] Input: Username, Password

[1594] Output: Sending authentication information

[1595] Step 3:

[1596] The server receives the entered information and authenticates it by checking it against a database.

[1597] Specific operation: The server receives the authentication information sent from the terminal and checks it against the user data in the database (MySQL or PostgreSQL) using an SQL query to confirm that it matches.

[1598] Input: Username, Password

[1599] Output: Authentication result

[1600] Step 4:

[1601] The server sends the authentication result to the terminal, notifying it of the success or failure of the login.

[1602] Specific operation: The server formats the authentication result as an HTTP response and sends it to the terminal. The terminal receives the result and redirects the user to the virtual museum screen if the login is successful, or displays an error message if the login is unsuccessful.

[1603] Input: Authentication result

[1604] Output: Login success message or error message

[1605] Virtual space display and operation processing

[1606] Step 1:

[1607] The server sends the virtual museum data to the terminal.

[1608] Specific operation: The server generates the 3D model data (JSON or XML format) required for the virtual museum and sends it to the terminal.

[1609] Input: Request after successful login

[1610] Output: 3D model data

[1611] Step 2:

[1612] The device renders the received data and displays the virtual space to the user.

[1613] Specific operation: The device renders the received 3D model data using graphics libraries such as WebGL or Three.js, and displays the virtual space on the browser.

[1614] Input: 3D model data

[1615] Output: Display of virtual space

[1616] Step 3:

[1617] Users move around the virtual space and view the exhibits.

[1618] Specific actions: Users navigate the virtual space using a keyboard, mouse, or touchscreen, and can click on specific exhibits to view more information.

[1619] Input: User actions

[1620] Output:Detailed information about the exhibit

[1621] Interactive tour and sentiment analysis processing

[1622] Step 1:

[1623] Users enter questions and comments about the artwork via text on their device.

[1624] Specific behavior: The user enters a question or comment in the text box and clicks the submit button.

[1625] Input: Text data (questions and comments)

[1626] Output: Sending text data

[1627] Step 2:

[1628] The server analyzes the content of the speech using natural language processing algorithms.

[1629] Specific operation: The server receives the text data, analyzes it using a natural language processing engine (Google NLP API, SpaCy, NLTK), and infers the user's emotions and intentions.

[1630] Input: Text data

[1631] Output: Analysis results (emotions and intentions)

[1632] Step 3:

[1633] The server estimates the user's emotions and intentions and generates an appropriate response.

[1634] Specific operation: The server generates an appropriate response using a generative AI model (OpenAI GPT-3) based on the analysis results.

[1635] Input: Analysis results

[1636] Output: Response text

[1637] Step 4:

[1638] The server sends the response to the terminal and displays it to the user.

[1639] Specific operation: The server formats the generated response text as an HTTP response and sends it to the terminal, which then displays the response text to the user.

[1640] Input: Response text

[1641] Output: View the response

[1642] Handling online events and workshops

[1643] Step 1:

[1644] The server manages and notifies schedules for online events and workshops.

[1645] Specific operation: The server stores the schedule of the online event in a database and notifies participating users via email or other means.

[1646] Input: Online event information

[1647] Output: Notification message

[1648] Step 2:

[1649] The user checks the event information on their device and applies to participate.

[1650] Specific operation: The user checks the event information on the device and presses the participation application button.

[1651] Input: Event participation application

[1652] Output: Send application information

[1653] Step 3:

[1654] The server manages the participant list and controls participant access at the start of the event.

[1655] Specific operation: The server manages the participant list, creates a virtual room at the start of the event, and invites participants.

[1656] Input: Participant list

[1657] Output: Virtual room URL

[1658] Step 4:

[1659] Users join events on their devices and interact with other participants through avatars.

[1660] Specific operation: A user accesses the URL of a virtual room and interacts with other participants through their avatars.

[1661] Input: Virtual room URL

[1662] Output: Interaction through avatars

[1663] (Application example 1)

[1664] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1665] Modern retail experiences require users to explore a wide variety of products without physical constraints and to receive real-time information to support purchasing decisions. However, existing online shopping systems require a lot of effort for users to obtain detailed product information and receive appropriate recommendations, and they face challenges in providing personalized responses based on the user's emotions and intent. Furthermore, there is a lack of an environment for effectively hosting new product launches and fashion events online.

[1666] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1667] In this invention, the server includes: a means for building a community using avatars in a virtual space and providing online events and workshops; a means for analyzing users' natural language utterances and estimating their emotions; a means for an AI curator to instantly respond to users' questions and provide information; a means for updating display content based on the user's movements and actions in the virtual store; a means for searching for items in the virtual store in natural language using a smartphone or head-mounted display and displaying the results; a means for making real-time suggestions for recommended items and outfits; and a means for managing and notifying the schedule of online fashion events and workshops. This allows users to freely move around the virtual store using their own avatars, efficiently explore a variety of products, and receive personalized information and recommendations in real time. Furthermore, it also becomes possible to effectively hold new product launches and fashion events online, significantly improving the user experience.

[1668] "Virtual space" refers to a digital environment in which users can operate in a virtual realm that is different from their actual physical environment.

[1669] An "avatar" is a digital character with which a user interacts in a virtual space.

[1670] A "community" is a group of users who share a common purpose or interest and interact with each other.

[1671] An "online event" is a real-time event or occasion held over the Internet.

[1672] A "workshop" is a participatory educational and training setting for learning specific skills and knowledge.

[1673] "Natural language" refers to the language used by humans on a daily basis, which needs to be converted into a form that is easy for computers to understand.

[1674] "Emotion" is a concept that represents a human mental state, such as joy, sadness, or anger.

[1675] "AI Curator" refers to a system that uses artificial intelligence to respond to user questions and provide information.

[1676] "Document" does not necessarily mean text data, but may also include drawings, explanatory text, diagrams, etc.

[1677] "Display content" is the collection of information that a user can visually see on the screen.

[1678] A "smartphone" is a portable information terminal that can use a variety of applications in addition to the functions of a mobile phone.

[1679] A "head-mounted display" is a display device worn on the head, and is primarily used to experience virtual reality and augmented reality.

[1680] "Items" are goods or merchandise that users can view or purchase within the virtual space.

[1681] "Recommendation" refers to the presentation of specific products or information based on the user's preferences and behavior.

[1682] "Coordination suggestions" refers to providing recommendations on how to effectively combine fashion items.

[1683] "Managing your schedule" refers to scheduling and coordinating specific events and tasks.

[1684] "Notification" refers to the sending of a message to a user informing them of important information or the onset of an event.

[1685] This invention is a system that allows users to use avatars in a virtual space to explore products and engage in two-way communication, such as asking questions and sending comments. Specifically, using devices such as smartphones or head-mounted displays, users can access a virtual store, search for and view items in real time, and obtain information about artworks.

[1686] System overview and basic processing

[1687] The virtual fashion boutique system includes a smartphone, a head-mounted display, and a server as its main hardware and software components. Specific software components include a natural language processing engine and a sentiment analysis engine.

[1688] User authentication and login

[1689] The server accepts system startup and provides a page for user authentication. Using a terminal, the user accesses the login screen and enters their username and password. The server receives the entered information and performs authentication by comparing it with a database. If authentication is successful, the server sends the virtual store data to the terminal and notifies it that login was successful.

[1690] Displaying and manipulating virtual space

[1691] The server sends the virtual boutique data to the device, which then renders the received data and displays the virtual space to the user, allowing the user to move around the virtual space and view the exhibited products.

[1692] Natural Language Search and Sentiment Analysis

[1693] Users input questions and opinions about fashion items via text. The device sends the input data to the server, which then analyzes the text using a natural language processing engine. The server then infers the user's emotions and intentions, generates an appropriate response, and sends it to the device.

[1694] Display of product information and AI curator response

[1695] When a user stops in front of a specific item, the server generates detailed information related to that item and displays it to the user. The AI ​​curator instantly responds to the user's questions and provides relevant product information.

[1696] Hosting online events and workshops

[1697] The server schedules new product launches and fashion events and notifies users. Users use their devices to check event information and register for participation. The server manages the participant list and controls participant access when the event starts. The device displays the event in real time and allows users to interact with other participants.

[1698] Specific examples

[1699] Example 1: User login and virtual space display

[1700] Scenario 1: A user logs into a virtual fashion boutique

[1701] The user accesses the login screen on their smartphone and enters their username and password. The server receives the user's authentication information and compares it with the user data in the database. If authentication is successful, the server sends the virtual boutique data to the device along with a "Login successful" message. The device then renders the received data and displays the virtual space to the user.

[1702] Example 2: User questions and AI curator responses

[1703] Scenario 2: User asks about a specific dress

[1704] The user moves through the virtual space and stops in front of a particular dress. The user types "What is the trend for this dress?" into a text box. The device sends the input data to the server, which analyzes the text using a natural language processing engine. Based on the analysis results, the server generates trend information for the dress and returns a response to the user. The user checks the response and deepens their knowledge about the dress.

[1705] Example 3: Participating in an online fashion event

[1706] Scenario 3: User attends an online fashion event

[1707] The server notifies users of the schedule for a new online fashion event, "Introducing the Latest Trends." Users check the event information on their devices and apply to participate. When the event starts, the server creates a virtual room and invites participants. Users access the virtual room and participate in the event while interacting with other participants.

[1708] Prompt Sentence Examples

[1709] A user logs into a fashion boutique and asks, "Tell me about the latest dress trends." An AI curator then answers the question, and the user is invited to a new fashion event.

[1710] With the above-described configuration, the present invention realizes a virtual store where a wide variety of products can be efficiently and effectively searched for without being restricted by physical movement or time.

[1711] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1712] Step 1:

[1713] The server accepts the system startup and provides a page for user authentication.

[1714] Input: User's access request

[1715] Data calculation: Display of user authentication page

[1716] Output:Login screen

[1717] Step 2:

[1718] The user accesses the login screen on the device and enters their username and password.

[1719] Input: Username, Password

[1720] Data Computation: Input Data Collection and Formatting

[1721] Output: Authentication data

[1722] Step 3:

[1723] The server receives the entered information and authenticates it by checking it against a database.

[1724] Input: Authentication data

[1725] Data calculation: database matching, authentication calculation

[1726] Output: Authentication result

[1727] Step 4:

[1728] The server sends the authentication result to the terminal, notifying it of the success or failure of the login.

[1729] Input: Authentication result

[1730] Data Calculation: Message Generation

[1731] Output: Notification of successful or failed login

[1732] Step 5:

[1733] The server sends the virtual boutique data to the terminal.

[1734] Input: Login success notification

[1735] Data calculation: data packaging, transmission processing

[1736] Output: Virtual boutique data

[1737] Step 6:

[1738] The device renders the received data and displays the virtual space to the user.

[1739] Input: Virtual Boutique Data

[1740] Data calculation: data rendering, graphics processing

[1741] Output: Virtual space display

[1742] Step 7:

[1743] Users move around the virtual space and view the exhibits.

[1744] Input: User Action

[1745] Data calculation: Position update, display update

[1746] Output: Updated display

[1747] Step 8:

[1748] The user enters a text question about a particular item.

[1749] Input: User text input

[1750] Data calculation: Collecting and structuring input data

[1751] Output: Question data

[1752] Step 9:

[1753] The terminal sends the input data to the server.

[1754] Input: Question data

[1755] Data calculation: Data transmission processing

[1756] Output: Data sent

[1757] Step 10:

[1758] The server analyzes the text using a natural language processing engine.

[1759] Input: Data sent

[1760] Data Computing: Natural Language Analysis, Text Semantic Understanding

[1761] Output: Analysis results

[1762] Step 11:

[1763] The server estimates the user's emotions and intentions, generates an appropriate response, and sends it to the device.

[1764] Input: Analysis results

[1765] Data calculation: emotion estimation, response generation

[1766] Output: Generated response data

[1767] Step 12:

[1768] When a user stops in front of a particular item, the server generates and displays detailed information related to that item.

[1769] Input: Item stop event

[1770] Data Calculations: Finding and Generating More Information

[1771] Output: Show detailed information

[1772] Step 13:

[1773] The server manages the schedule of new product launches and fashion events and notifies users.

[1774] Input: Event schedule information

[1775] Data calculation: Schedule management, notification generation

[1776] Output: Notification message

[1777] Step 14:

[1778] The user checks the event information on the device and applies to participate.

[1779] Input: Event participation application

[1780] Data processing: collection and transmission of application data

[1781] Output: Participation application data

[1782] Step 15:

[1783] The server manages the participant list and controls participant access at the start of the event.

[1784] Input: Application data

[1785] Data operations: list management, access control

[1786] Output: Participant access control

[1787] Step 16:

[1788] The device displays the event in real time and allows users to interact with other participants.

[1789] Input: Real-time event data

[1790] Data calculation: Event display and AC function provided

[1791] Output: Real-time event display, interaction function

[1792] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1793] This invention relates to a system for providing an art appreciation experience in a virtual space, and in particular to a combined emotion engine that recognizes user emotions. Specifically, it includes functions for building an avatar community and hosting online events and workshops, analyzing users' natural language utterances to estimate their emotions, providing information about artworks through an AI curator that responds to users' questions in real time, and updating the display content based on the user's movements and actions in the virtual museum.

[1794] System overview and basic processing

[1795] The virtual museum system works in conjunction with a server and terminals. The server is responsible for data management and processing, while the terminals function as the user interface. Furthermore, an emotion engine is built in, which analyzes user emotion data and provides feedback to the entire system.

[1796] User authentication and login

[1797] 1. The server accepts the system startup and provides a page for user authentication.

[1798] 2. The user accesses the login screen on the device and enters their username and password.

[1799] 3. The server receives the entered information and authenticates it by checking it against a database.

[1800] 4. The server sends the authentication result to the terminal and notifies it of the success or failure of the login.

[1801] Displaying and manipulating virtual space

[1802] 1. The server sends the virtual museum's 3D model and artwork data to the user who has successfully logged in.

[1803] 2. The device processes the received data using a rendering engine and displays the virtual space on the user's display.

[1804] 3. Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[1805] Interactive Tours and Sentiment Analysis

[1806] 1. Users stop in front of a particular piece of art and type their question or comment into a text box.

[1807] 2. The terminal sends the entered text data to the server.

[1808] 3. The server analyzes the received text data using a natural language processing engine and uses an emotion engine to infer the user's intentions and emotions.

[1809] 4. The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[1810] 5. The server generates a response and sends it to the terminal, where it displays it to the user.

[1811] Hosting online events and workshops

[1812] 1. The server notifies registered users of new online events and workshops scheduled.

[1813] 2. The device displays the received notification to the user, and the user checks the event information.

[1814] 3. If the user wishes to register, he / she sends a participation application request to the server from his / her terminal.

[1815] 4. The server accepts the participation application and registers it on the participant list.

[1816] 5. At the start of the event, the server creates a virtual room for participants and sends them an access link.

[1817] 6. Users use their devices to access the virtual room and participate in online events and workshops.

[1818] 7. Users can interact with other participants through their avatars, chat and exchange opinions.

[1819] Specific examples

[1820] Example 1: User login and sentiment engine analysis

[1821] Scenario 1: User A logs in to the virtual museum

[1822] 1. User A accesses the login screen on the device and enters their username and password.

[1823] 2. The server receives User A's authentication information and checks it against the user data in its database.

[1824] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[1825] 4. The device renders the received data and displays the virtual space to User A.

[1826] Example 2: User Questions and Emotion Engine Responses

[1827] Scenario 2: User B asks about a specific painting

[1828] 1. User B moves through the virtual space and stops in front of a specific painting.

[1829] 2. User B types "What is the background of this picture?" into the text box.

[1830] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms and an emotion engine.

[1831] 4. Based on the analysis results, the server generates customized information based on the painting's background information and the user's emotions, and returns a response to User B.

[1832] 5. User B reviews the response and learns more about the painting.

[1833] Example 3: Online workshops and the use of sentiment analysis

[1834] Scenario 3: User C attends an online workshop

[1835] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[1836] 2. User C checks the event information on their device and applies to participate.

[1837] 3. At the start of the event, the server creates a virtual room and invites participants.

[1838] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[1839] 5. The server analyzes users' comments in real time during the workshop using an emotion engine and provides interactive content that responds to participants' emotional changes.

[1840] As explained above, by combining an emotion engine that recognizes the user's emotions, the present invention can provide a more personalized art experience and deepen the user's appreciation experience, allowing the user to enjoy a deeper art experience without being restricted by physical constraints.

[1841] The processing flow will be explained below.

[1842] User login process

[1843] Step 1:

[1844] The user operates the device and accesses the login screen, enters their username and password, and clicks the "Login" button.

[1845] Step 2:

[1846] The terminal generates a request to send the entered user name and password to the server, and sends it to the server.

[1847] Step 3:

[1848] The server checks the received username and password against a database to see if the user is valid.

[1849] Step 4:

[1850] The server generates an authentication result and sends a success or failure message to the terminal.

[1851] Step 5:

[1852] The terminal receives the authentication result and displays a login success message to the user, or an error message if authentication fails.

[1853] Displaying and manipulating virtual space

[1854] Step 1:

[1855] The server prepares to send the virtual museum's 3D models and artwork data to users who have successfully logged in.

[1856] Step 2:

[1857] The server sends the virtual museum data to the terminal.

[1858] Step 3:

[1859] The device processes the virtual museum data received using a rendering engine and displays the virtual space on the user's display.

[1860] Step 4:

[1861] Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[1862] Step 5:

[1863] The device detects the user's actions and reflects the corresponding actions (movement, change of viewpoint, selection of specific objects) in the virtual space in real time.

[1864] Interactive Tours and Sentiment Analysis

[1865] Step 1:

[1866] Users stop in front of a particular piece of art and type their question or comment into a text box.

[1867] Step 2:

[1868] The terminal generates a request for transmitting the input text data to the server, and transmits the request to the server.

[1869] Step 3:

[1870] The text data received by the server is analyzed using a natural language processing engine, and the emotion engine is used to infer the user's intentions and emotions.

[1871] Step 4:

[1872] The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[1873] Step 5:

[1874] The server sends the generated response data to the terminal.

[1875] Step 6:

[1876] The terminal displays the received response data to the user, who then checks the detailed information.

[1877] Hosting online events and workshops

[1878] Step 1:

[1879] The server generates data to notify registered users of new online event and workshop schedules.

[1880] Step 2:

[1881] The server sends the notification data to the registered user's terminal.

[1882] Step 3:

[1883] The device displays the received notification to the user, who then checks the event information.

[1884] Step 4:

[1885] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[1886] Step 5:

[1887] The server accepts the participation application and registers it in the participant list.

[1888] Step 6:

[1889] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[1890] Step 7:

[1891] Users use their devices to access virtual rooms and participate in online events and workshops.

[1892] Step 8:

[1893] Users interact with other participants through their avatars, chatting and exchanging opinions.

[1894] Step 9:

[1895] During the workshop, the server analyzes users' comments in real time using an emotion engine and provides interactive content that responds to participants' emotional changes.

[1896] Through the above processing steps, the virtual museum system provides users with an immersive art experience and knowledge acquisition, and by combining it with an emotion engine, it can realize a personalized experience based on the user's emotions.

[1897] Example 2

[1898] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1899] Virtual art appreciation lacks a personalized experience that takes into account the user's emotions. This can lead to a decrease in user satisfaction and motivation to participate. Furthermore, while users need fast and accurate responses to their questions, existing systems lack immediacy and adaptability.

[1900] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for building a community using avatars in a virtual space and providing online events and workshops; a means for analyzing the content of users' natural language utterances and estimating the user's emotions; a means for an AI curator to instantly respond to users' questions and provide information about artworks; a means for updating display content based on the user's movements and actions in the virtual museum; a means for performing user authentication and login processing; a means for displaying the virtual space using a user interface rendering engine; a means for analyzing input text data using a natural language processing engine and estimating the user's intentions and emotions using an emotion engine; and a means for generating response data based on the analysis results and displaying it to the user. This enables a personalized art appreciation experience that takes user emotions into consideration, thereby improving user satisfaction and willingness to participate. It also enables rapid and accurate responses, providing a more interactive and engaging experience.

[1901] A "virtual space" is a virtual environment generated using computers and virtual reality technology that users can experience through their senses of sight, hearing, touch, etc.

[1902] An "avatar" is a character or alter ego that a user uses to represent themselves in a virtual space.

[1903] A "community" is a group of users who come together under a common purpose or interest and interact and cooperate with each other.

[1904] An "online event" is a gathering or event that takes place in real time over the internet.

[1905] A "workshop" is an event held for participants to learn about a particular topic or technology and share practical experience.

[1906] "User authentication" means the process of verifying that a user attempting to access a system is a registered and authorized user.

[1907] A "natural language processing engine" is a computer algorithm or program that analyzes, understands, and generates the natural language used by humans on a daily basis.

[1908] An "emotion engine" is an algorithm or program that analyzes and infers a user's emotions from their text or voice data.

[1909] An "artificial intelligence curator" is a system that uses AI technology to provide information in response to user questions and operations, and to explain artworks.

[1910] A "virtual museum" is a museum in a virtual space set up on the Internet, where digital artworks and exhibits can be viewed.

[1911] A "rendering engine" is a program or software that generates computer graphics and displays them on a display.

[1912] A "session" refers to a series of operations and communications between when a user logs in to the system and when they log out, and is a temporary unit of communication that is specific to each user.

[1913] This invention relates to a system for providing an art appreciation experience in a virtual space, particularly incorporating an emotion engine that recognizes the user's emotions. Users can access the virtual space using an avatar to view exhibits and participate in online events and workshops. It also has the ability to analyze the user's comments and generate appropriate responses based on their emotions.

[1914] System configuration

[1915] This system works in conjunction with a server and a device. The server manages and processes data, while the device functions as the user interface. It is equipped with an emotion engine and a natural language processing engine, which analyzes user emotions to provide a personalized experience. Specifically, it includes the following components:

[1916] 1. Server

[1917] Database Management System (DBMS): Manages user information and artwork data. For example, MySQL or PostgreSQL can be used.

[1918] Natural language processing engine: Analyzes user utterances. For example, GPT-4 is used.

[1919] Emotion engine: Estimates user emotions.

[1920] Rendering engine: Processes the 3D models to generate the virtual space. For example, Unity or Unreal Engine is used.

[1921] 2. Terminal

[1922] User Interface (UI): Provides access to and control of virtual spaces, including PCs and VR devices.

[1923] Input devices: keyboard, mouse, VR controller, etc.

[1924] 3. Users

[1925] Avatar: A character or alter ego that a user controls in a virtual space.

[1926] Specific processing details

[1927] User authentication and login

[1928] The server starts the system and provides a user authentication page. The user enters their username and password on the terminal and sends them to the server. The server checks the database for authentication and returns the result to the terminal.

[1929] Displaying and manipulating virtual space

[1930] After a successful login, the server sends data on the virtual museum's 3D model and artworks. The device receives the data and renders it using a rendering engine. The user then uses a mouse or VR controller to move around the virtual space.

[1931] Interactive Tours and Sentiment Analysis

[1932] Users stop in front of a particular artwork and enter their questions or comments in text. The device sends the data to a server, where it is analyzed by a natural language processing engine. The emotion engine infers the user's intentions and emotions, and the server generates an appropriate response and sends it to the device.

[1933] Hosting online events and workshops

[1934] The server notifies users of new event and workshop schedules. Users can check the schedules on their devices and apply to participate. The server registers participants on a list and opens a virtual room at the start of the event. Users can access the room and interact with other participants.

[1935] Specific scenarios

[1936] Scenario 1: User Logged In

[1937] User A accesses the login screen on their device and enters their username and password. The server receives the authentication information and checks it against the database. If authentication is successful, the virtual museum data is sent to the device and the virtual space is displayed to User A.

[1938] Scenario 2: User Questions and Sentiment Analysis

[1939] User B asks a question about a particular painting. He types, "What is the background of this painting?" into his device. The text data is sent to the server, where it is analyzed by the natural language processing engine and emotion engine. The server generates a response based on the analysis results and sends it to the device.

[1940] Scenario 3: Participating in an online workshop

[1941] User C receives the schedule for a new online workshop and applies to participate. The server registers the participants in the list and generates a virtual room at the start of the event. User C accesses the room and participates in the workshop.

[1942] Prompt Sentence Examples

[1943] "When a user stands in front of a particular painting and asks, 'What is the background to this painting?', provide context for the artwork based on the user's emotions."

[1944] Through these functions, the present invention can recognize the user's emotions and provide a personalized art appreciation experience, allowing the user to enjoy a deep art experience without being restricted by physical constraints.

[1945] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1946] The flow of this system's program processing

[1947] User authentication and login

[1948] Step 1:

[1949] The server accepts the system startup, provides a user authentication page, generates a URL for the user to access the login screen, and returns it as an HTTP response.

[1950] Input: User's access request

[1951] Output: User authentication page

[1952] Specific behavior:

[1953] The server receives the HTTP request and prepares the HTML for the user authentication page.

[1954] The server returns the HTML as an HTTP response.

[1955] Step 2:

[1956] The user accesses the login screen on the device and enters their username and password.

[1957] Input: Username and Password

[1958] Output: Login credentials

[1959] Specific behavior:

[1960] The user enters their username and password on the login page and clicks the submit button.

[1961] The device sends the login credentials to the server.

[1962] Step 3:

[1963] The server receives the entered information, compares it with the database, and performs authentication. It generates an authentication result and sends it to the terminal.

[1964] Input: Login credentials

[1965] Output: Authentication result

[1966] Specific behavior:

[1967] The server checks the username and password against a database query.

[1968] Based on the result of the comparison, the success or failure of the authentication is determined, and an authentication result message is generated.

[1969] An authentication result message is sent to the terminal.

[1970] Displaying and manipulating virtual space

[1971] Step 4:

[1972] After the server successfully logs in, it sends the 3D model of the virtual museum and data on the artworks.

[1973] Input: Login success notification

[1974] Output: 3D model data, art work data

[1975] Specific behavior:

[1976] The server retrieves the virtual museum's 3D model data and artwork data from the database.

[1977] The data is packaged and sent to the device.

[1978] Step 5:

[1979] The data received by the terminal is processed using a rendering engine to display the virtual space.

[1980] Input: 3D model data, art work data

[1981] Output: Rendered image of the virtual space

[1982] Specific behavior:

[1983] The device reads the data it receives and inputs it into a rendering engine such as Unity or Unreal Engine.

[1984] The rendering engine draws the 3D model and displays it on the device's display.

[1985] Step 6:

[1986] Users use a mouse, keyboard, or VR controller to move around the virtual space and view the exhibits.

[1987] Input: User Action

[1988] Output: Movement within the virtual space and change of viewpoint of exhibits

[1989] Specific behavior:

[1990] The user operates an input device (mouse, keyboard, VR controller).

[1991] The device captures input events and sends instructions to the rendering engine.

[1992] The rendering engine updates the screen and moves the user's viewpoint and position.

[1993] Interactive Tours and Sentiment Analysis

[1994] Step 7:

[1995] Users stop in front of a particular piece of art and type their question or comment into a text box.

[1996] Input: User text input

[1997] Output: Input text data

[1998] Specific behavior:

[1999] The user types a question into the text box and clicks the submit button.

[2000] The terminal transmits the input text data to the server.

[2001] Step 8:

[2002] The text data received by the server is analyzed using a natural language processing engine, and the emotion engine is used to infer the user's intentions and emotions.

[2003] Input: Input text data

[2004] Output: analysis results, emotion estimation results

[2005] Specific behavior:

[2006] The server inputs the input text into a natural language processing engine (GPT-4).

[2007] A natural language processing engine analyzes the text, and an emotion engine infers emotions.

[2008] Generate analysis results and emotion estimation results.

[2009] Step 9:

[2010] The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[2011] Input: Analysis results, emotion estimation results

[2012] Output: Customized response data

[2013] Specific behavior:

[2014] The server creates a response message based on the analysis results.

[2015] The response message is customized with emotional information to generate final response data.

[2016] Step 10:

[2017] The response data generated by the server is sent to the terminal and displayed to the user.

[2018] Input: Customized response data

[2019] Output: Display of response message

[2020] Specific behavior:

[2021] The server sends the customized response data to the terminal.

[2022] The terminal receives the response data and displays it to the user.

[2023] Hosting online events and workshops

[2024] Step 11:

[2025] The server notifies registered users of new online events and workshop schedules.

[2026] Input: Event Schedule

[2027] Output: Notification message

[2028] Specific behavior:

[2029] The server generates a new event schedule.

[2030] Notify registered users of event schedules.

[2031] Step 12:

[2032] The device displays the received notification to the user, who then checks the event information.

[2033] Input: Notification message

[2034] Output: Display of event information

[2035] Specific behavior:

[2036] The terminal receives the notification message.

[2037] The device displays the event information to the user.

[2038] Step 13:

[2039] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[2040] Input: User participation application

[2041] Output: Application request

[2042] Specific behavior:

[2043] The user clicks the sign up button.

[2044] The terminal transmits the participation application data to the server.

[2045] Step 14:

[2046] The server accepts the participation application and registers it in the participant list.

[2047] Input: Application request

[2048] Output: Update participant list

[2049] Specific behavior:

[2050] The server receives the subscription request.

[2051] Add the user to the participant list and save the data.

[2052] Step 15:

[2053] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[2054] Input: Participant list

[2055] Output: Virtual room access link

[2056] Specific behavior:

[2057] The server generates a virtual room at the start of the event.

[2058] Send participants the virtual room access link.

[2059] Step 16:

[2060] Users use their devices to access virtual rooms and participate in online events and workshops.

[2061] Input: Virtual room access link

[2062] Output: Joining a virtual room

[2063] Specific behavior:

[2064] The user clicks on the access link and enters the virtual room.

[2065] The device displays the virtual room and helps you participate in the event.

[2066] Step 17:

[2067] Users interact with other participants through their avatars, chatting and exchanging opinions.

[2068] Input: User interaction intent

[2069] Output: Chat messages, exchange of opinions

[2070] Specific behavior:

[2071] Users control avatars and interact with other participants.

[2072] The device sends chat messages and exchanges of opinions to the other party in real time.

[2073] Prompt Sentence Examples

[2074] "When a user stands in front of a particular painting and asks, 'What is the background to this painting?', provide context for the artwork based on the user's emotions."

[2075] (Application example 2)

[2076] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2077] Conventional virtual community systems and online shopping systems are insufficient in providing personalized experiences based on user emotions, making it difficult to improve user satisfaction. Furthermore, it is difficult to provide product suggestions and customized information based on real-time emotion analysis, making it difficult to stimulate users' purchasing motivation.

[2078] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for building a community using avatars in a virtual space and providing online events and workshops, a means for analyzing the content of users' natural language utterances and estimating the users' emotions, a means for an AI curator to immediately respond to users' questions and provide product information, a means for updating display content based on the users' movements and actions in the virtual store, and a means for providing personalized product suggestions and explanations based on the users' emotions. This makes it possible to provide users with personalized experiences and product suggestions in real time.

[2079] A "virtual space" is a digital environment distinct from the real world, in which users can engage in activities and interactions through avatars.

[2080] An "avatar" is a character or icon that represents a user in a virtual space and acts in place of the user's actions and communication.

[2081] A "community" is a group formed by users with common interests and goals, and is a place for interaction and information sharing within a virtual space.

[2082] An "online event" is an event held in a virtual space via the Internet, in which many users can participate simultaneously.

[2083] A "workshop" is a session for learning knowledge or skills on a specific topic, and is conducted in a virtual space.

[2084] "The user's natural language utterances" refer to the text data or voice data entered or spoken by the user, which is what the system analyzes.

[2085] "Means for estimating emotions" refers to technical means for determining a user's emotional state from their natural language and facial expressions, and includes emotion analysis engines, etc.

[2086] An "AI curator" is a virtual entity within the system that uses artificial intelligence to generate appropriate responses to user questions and provide information.

[2087] "Product information" refers to detailed descriptions, usage instructions, comparison information, etc., of the products and services available in the virtual store.

[2088] A "virtual store" is a virtual store available via the Internet where users can browse, select, and purchase products.

[2089] "Movements and actions" refer to the various actions and operations that users perform within the virtual space, and the system reacts based on these actions and updates the displayed content.

[2090] "Means for updating display content" refers to technical means for dynamically changing the information displayed in the virtual space in response to the user's movements and actions.

[2091] "Personalized product suggestions" is a system function that individually recommends appropriate products and services based on the user's emotions and past behavioral data.

[2092] A "natural language processing engine" is a program that analyzes text entered by a user and understands its content and intent.

[2093] A "means for generating a response" is a technical means that has the function of generating appropriate answers or information based on the user's emotions and the content of the question.

[2094] "Detailed Information" includes detailed explanations and background information about specific products or content, and is provided upon request by the user.

[2095] The present invention relates to a virtual store system that builds a community using avatars in a virtual space and has the ability to recognize user emotions. This system offers online events and workshops, analyzes user comments, provides instant responses from AI curators, updates display content based on user movements and actions, and provides personalized product suggestions and explanations. The following describes the specific configuration and functions required to implement the present invention.

[2096] First, the basic configuration of the system involves linking a server and a terminal. The server is responsible for data management and processing, while the terminal functions as the user interface. This allows users to access the virtual space using their own terminal and use various services.

[2097] The server includes an "emotion analysis engine" that analyzes users' natural language utterances to estimate their emotions, and an "AI curator" that responds instantly to users' questions. The emotion analysis engine collects users' text and voice data and identifies their emotions using machine learning algorithms. This analysis uses a deep learning model trained using, for example, Keras.

[2098] Next, when the user logs in, the device receives the 3D model of the virtual store and product data from the server and displays the virtual space using a rendering engine. The user can move around the virtual space using a mouse, keyboard, or VR controller to view and select displayed products.

[2099] As a concrete example, consider the case where a user stops in front of a specific product. When the user enters a question about the product, the device sends the text data to the server, where the intent and sentiment of the question are analyzed using a sentiment analysis engine and natural language processing engine. Based on the analysis results, the AI ​​curator generates appropriately customized detailed information about the product and responds to the user. For example, if a user asks, "What is this product?" the system might respond, "This product is the latest model that was recently released. It features a particularly high-performance battery and is popular with many users."

[2100] For online events and workshops, the server notifies registered users of new event information and accepts participation registrations. When the event begins, a virtual room is created and users can participate in real time. At this time, the conversation data between users is also analyzed by the emotion analysis engine, and the system provides interactive content according to their emotions.

[2101] In this way, the system of the present invention can recognize the user's emotions in a virtual store and provide a more advanced and personalized shopping experience.

[2102] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2103] Step 1:

[2104] The user accesses the login screen on the terminal and enters the username and password. The input data (username and password) is sent to the server. The server compares the received data with the user database, determines whether the authentication was successful or not, and returns the result to the terminal. The terminal displays the login result to the user.

[2105] Step 2:

[2106] The server sends a 3D model of the virtual store and product data to the user who has been successfully authenticated. The device processes the received data using a rendering engine (e.g., Unity) and displays it as a virtual space on the user's display. The user can move around the virtual space using a mouse, keyboard, or VR controller and view the products on display.

[2107] Step 3:

[2108] The user stops in front of a specific product and enters a question or comment into a text box. The device sends the entered text data to the server. The server then analyzes the received text data using a natural language processing engine (e.g., Transformers' BERT model) to understand the content and intent of the input text. It also uses a sentiment analysis engine (e.g., a deep learning model using Keras) to determine the user's emotional state.

[2109] Step 4:

[2110] Based on the analysis results, the server generates appropriate response data from the AI ​​curator that corresponds to the user's sentiment and question. This response data includes detailed product information and personalized suggestions. The server then sends the generated response data to the device, which then displays it to the user.

[2111] Step 5:

[2112] If the user continues to move around the virtual space and shows interest in other products or areas, the server receives movement and action data and dynamically updates the content displayed in the virtual space, adjusting the display to show products and related information that the user is likely to be interested in next.

[2113] Step 6:

[2114] The server periodically notifies users of the schedule of online events and workshops. When a user wishes to register for participation, they send a request to the server from their device. The server accepts the registration, creates a virtual room, and sends an access link to participants when the event starts. Users access the virtual room and participate in the event while interacting with other participants.

[2115] Step 7:

[2116] During online events and workshops, the server analyzes users' comments and behavioral data in real time and provides interactive content that responds to changes in their emotions. For example, if a user shows interest in a particular topic, additional details or related content will be displayed.

[2117] In this way, this system allows the server and terminals to work together, analyzing user emotions in real time and providing personalized shopping experiences and online event experiences.

[2118] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2119] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2120] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2121] [Fourth embodiment]

[2122] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2123] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2124] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2125] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2126] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2127] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2128] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2129] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2130] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2131] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2132] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2133] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2134] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2135] This invention is a system for providing an art appreciation experience in a virtual space. Specifically, it builds an avatar community and holds online events and workshops. It also includes a function to analyze the user's natural language comments and estimate their emotions. Furthermore, it has a function that allows an AI curator to instantly respond to the user's questions and provide information about the artworks.

[2136] System overview and basic processing

[2137] The virtual museum system works in cooperation with a server and terminals. The server is responsible for data management and processing, while the terminals function as the user interface.

[2138] User authentication and login

[2139] 1. The server accepts the system startup and provides a page for user authentication.

[2140] 2. The user accesses the login screen on the device and enters their username and password.

[2141] 3. The server receives the entered information and authenticates it by checking it against a database.

[2142] 4. The server sends the authentication result to the terminal and notifies it of the success or failure of the login.

[2143] Displaying and manipulating virtual space

[2144] 1. The server sends the virtual museum data to the terminal.

[2145] 2. The device renders the received data and displays the virtual space to the user.

[2146] 3. Users move around the virtual space and view the exhibits.

[2147] Interactive Tours and Sentiment Analysis

[2148] 1. The user enters questions or comments about the artwork in text on the device.

[2149] 2. The server analyzes the content of the speech using a natural language processing algorithm.

[2150] 3. The server infers the user's emotions and intentions and generates an appropriate response.

[2151] 4. The server sends the response to the terminal and displays it to the user.

[2152] Hosting online events and workshops

[2153] 1. The server manages and notifies schedules for online events and workshops.

[2154] 2. The user checks the event information on the device and applies to participate.

[2155] 3. The server manages the participant list and controls participant access when the event starts.

[2156] 4. Users can join the event on their devices and interact with other participants through their avatars.

[2157] Specific examples

[2158] Example 1: User login and virtual space display

[2159] Scenario 1: User A logs in to the virtual museum

[2160] 1. User A accesses the login screen on the device and enters their username and password.

[2161] 2. The server receives User A's authentication information and checks it against the user data in its database.

[2162] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[2163] 4. The device renders the received data and displays the virtual space to User A.

[2164] Example 2: User questions and AI curator responses

[2165] Scenario 2: User B asks about a specific painting

[2166] 1. User B moves through the virtual space and stops in front of a specific painting.

[2167] 2. User B types "What is the background of this picture?" into the text box.

[2168] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms.

[2169] 4. Based on the analysis results, the server generates background information for the painting and returns a response to User B.

[2170] 5. User B reviews the response and learns more about the painting.

[2171] Example 3: Participating in an online workshop

[2172] Scenario 3: User C attends an online workshop

[2173] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[2174] 2. User C checks the event information on their device and applies to participate.

[2175] 3. At the start of the event, the server creates a virtual room and invites participants.

[2176] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[2177] From the above explanation, it can be understood that the present invention provides a virtual museum experience that allows users to easily enjoy the charm of art even in today's busy world. This allows users to enjoy a deep art experience without being restricted by physical constraints.

[2178] The processing flow will be explained below.

[2179] User login process

[2180] Step 1:

[2181] The user operates the device and accesses the login screen, enters their username and password, and clicks the "Login" button.

[2182] Step 2:

[2183] The terminal generates a request to send the entered user name and password to the server, and sends it to the server.

[2184] Step 3:

[2185] The server checks the received username and password against a database to see if the user is valid.

[2186] Step 4:

[2187] The server generates an authentication result and sends a success or failure message to the terminal.

[2188] Step 5:

[2189] The terminal receives the authentication result and displays a login success message to the user, or an error message if authentication fails.

[2190] Displaying and manipulating virtual space

[2191] Step 1:

[2192] The server prepares to send the virtual museum's 3D models and artwork data to users who have successfully logged in.

[2193] Step 2:

[2194] The server sends the virtual museum data to the terminal.

[2195] Step 3:

[2196] The device processes the virtual museum data received using a rendering engine and displays the virtual space on the user's display.

[2197] Step 4:

[2198] Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[2199] Step 5:

[2200] The device detects the user's actions and reflects the corresponding actions (movement, change of viewpoint, selection of specific objects) in the virtual space in real time.

[2201] Interactive Tours and Sentiment Analysis

[2202] Step 1:

[2203] Users stop in front of a particular piece of art and type their question or comment into a text box.

[2204] Step 2:

[2205] The terminal generates a request for transmitting the input text data to the server, and transmits the request to the server.

[2206] Step 3:

[2207] The text data received by the server is analyzed using a natural language processing engine to infer the user's intentions and emotions.

[2208] Step 4:

[2209] The server generates appropriate response data based on the analysis results, including background information about the work, information about the creator, historical context, etc.

[2210] Step 5:

[2211] The server sends the generated response data to the terminal.

[2212] Step 6:

[2213] The terminal displays the received response data to the user, who then checks the detailed information.

[2214] Hosting online events and workshops

[2215] Step 1:

[2216] The server generates data to notify registered users of new online event and workshop schedules.

[2217] Step 2:

[2218] The server sends the notification data to the registered user's terminal.

[2219] Step 3:

[2220] The device displays the received notification to the user, who then checks the event information.

[2221] Step 4:

[2222] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[2223] Step 5:

[2224] The server accepts the participation application and registers it in the participant list.

[2225] Step 6:

[2226] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[2227] Step 7:

[2228] Users use their devices to access virtual rooms and participate in online events and workshops.

[2229] Step 8:

[2230] Users interact with other participants through their avatars, chatting and exchanging opinions.

[2231] Through the above processing steps, the virtual museum system provides users with a realistic art experience and the ability to acquire knowledge.

[2232] Example 1

[2233] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2234] Currently, many people find it difficult to visit museums and galleries due to physical constraints. However, online art appreciation experiences have limitations, and traditional methods do not allow users to enjoy a deep art experience or interact with other viewers. Furthermore, they lack the ability to provide relevant information about artworks and interactive responses based on the user's emotions. Given these factors, a new system is needed to provide a realistic art appreciation experience in a virtual space.

[2235] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2236] In this invention, the server includes means for transmitting a username and password entered from a terminal to the server for user authentication, comparing the username and password with a database and returning the authentication result, means for transmitting data of the virtual space from the server to the terminal and rendering and displaying the received data by the terminal, and means for the server to analyze the text data using a natural language processing engine when a user stops in front of a specific artwork in the virtual space and enters a question or comment, and generate and return an appropriate response. This allows users to enjoy a deep art experience and interact with other viewers without being restricted by physical constraints.

[2237] A "virtual space" is an imaginary space generated on a computer system that users can explore and manipulate as a 3D model.

[2238] An "avatar" is a digital character that users use to represent themselves in a virtual space.

[2239] An "online event" is a virtual event or gathering that takes place in real time over the Internet, allowing participants to share a common experience in a virtual space.

[2240] A "workshop" is an interactive learning or activity space based on a specific theme, where users can learn as participants by interacting with instructors and other participants.

[2241] "Natural language processing" is a technology that allows computers to analyze and understand human language (natural language), and is used to enable machines to understand and appropriately process user statements and input.

[2242] "Sentiment analysis" is a technique for inferring a user's emotional state from their speech or text, and is used to understand the user's current feelings and intentions.

[2243] "AI Curator" is a system based on artificial intelligence technology that acts as a virtual guide, providing instant information in response to user questions.

[2244] "User authentication" is the process of verifying the identity of a user by checking the credentials (username and password) entered by the user to access a system.

[2245] "Rendering" is the process by which a computer visually displays an image or video based on received 3D model data or other graphics data.

[2246] A "natural language processing engine" is a collection of algorithms and software that analyzes a user's text data, understands its content, and responds appropriately.

[2247] This invention is a system for providing an art appreciation experience in a virtual space. Specifically, it builds an avatar community and holds online events and workshops. It also includes a function to analyze the user's natural language comments and estimate their emotions. Furthermore, it has a function that allows an AI curator to instantly respond to the user's questions and provide information about the artworks.

[2248] System Configuration

[2249] This system operates in cooperation with a server and a terminal. The server is responsible for data management and processing, and the terminal functions as a user interface. The main components of this system and their functions are explained below.

[2250] User authentication and login

[2251] The server accepts system startup and provides a user authentication page. The user accesses the login screen on their terminal and enters their username and password. The server receives the entered information and performs authentication by checking it against a database (for example, MySQL or PostgreSQL). It sends the authentication result to the terminal and notifies them of the success or failure of the login. This process uses the web server Apache or Nginx.

[2252] Displaying and manipulating virtual space

[2253] The server sends the virtual museum data to the device, which then renders it using libraries such as WebGL and Three.js to display the virtual space to the user. The user can then move around the virtual space using a keyboard, mouse, or touchscreen to view the exhibits.

[2254] Interactive Tours and Sentiment Analysis

[2255] When a user enters a question or comment about an artwork into the device, the server analyzes the text data using a natural language processing algorithm (e.g., Google NLP API, SpaCy, or NLTK). The server estimates the user's emotions and intentions and generates an appropriate response. The response can be generated using a generative AI model (e.g., OpenAI GPT-3). The server then sends the generated response to the device and displays it to the user.

[2256] Hosting online events and workshops

[2257] The server manages and notifies schedules for online events and workshops. Users can check event information on their devices and apply to participate. The server manages the participant list and creates a virtual room at the start of the event to invite participants. Users can access the virtual room on their devices and interact with other participants through avatars.

[2258] Specific examples

[2259] The following are specific examples of this system:

[2260] Example 1: User login and virtual space display

[2261] When User A logs in to the virtual museum

[2262] 1. User A accesses the login screen on the device and enters their username and password.

[2263] 2. The server receives User A's authentication information and checks it against the user data in its database.

[2264] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[2265] 4. The device renders the received data and displays the virtual space to User A.

[2266] Example 2: User questions and AI curator responses

[2267] User B asks about a specific painting

[2268] 1. User B moves through the virtual space and stops in front of a specific painting.

[2269] 2. User B types "What is the background of this picture?" into the text box.

[2270] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms.

[2271] 4. The server generates background information for the painting based on the analysis results and returns a response to User B.

[2272] 5. User B reviews the response and learns more about the painting.

[2273] Example 3: Participating in an online workshop

[2274] User C participates in an online workshop

[2275] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[2276] 2. User C checks the event information on their device and applies to participate.

[2277] 3. At the start of the event, the server creates a virtual room and invites participants.

[2278] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[2279] Example prompt sentence:

[2280] "What is the background to this painting?"

[2281] "Tell me about Monet's works."

[2282] "When is the next online event?"

[2283] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2284] Handling user authentication and login

[2285] Step 1:

[2286] The server accepts the system startup and provides a page for user authentication.

[2287] Specific operation: When the server receives a startup signal, it uses a web server (Apache or Nginx) to generate an authentication page and sends the URL to the terminal.

[2288] Input: Start signal

[2289] Output: URL of authentication page

[2290] Step 2:

[2291] The user accesses the login screen on the device and enters their username and password.

[2292] Specific operation: The user uses a browser to access the authentication page displayed on the device, enters their username and password, and clicks the submit button.

[2293] Input: Username, Password

[2294] Output: Sending authentication information

[2295] Step 3:

[2296] The server receives the entered information and authenticates it by checking it against a database.

[2297] Specific operation: The server receives the authentication information sent from the terminal and checks it against the user data in the database (MySQL or PostgreSQL) using an SQL query to confirm that it matches.

[2298] Input: Username, Password

[2299] Output: Authentication result

[2300] Step 4:

[2301] The server sends the authentication result to the terminal, notifying it of the success or failure of the login.

[2302] Specific operation: The server formats the authentication result as an HTTP response and sends it to the terminal. The terminal receives the result and redirects the user to the virtual museum screen if the login is successful, or displays an error message if the login is unsuccessful.

[2303] Input: Authentication result

[2304] Output: Login success message or error message

[2305] Virtual space display and operation processing

[2306] Step 1:

[2307] The server sends the virtual museum data to the terminal.

[2308] Specific operation: The server generates the 3D model data (JSON or XML format) required for the virtual museum and sends it to the terminal.

[2309] Input: Request after successful login

[2310] Output: 3D model data

[2311] Step 2:

[2312] The device renders the received data and displays the virtual space to the user.

[2313] Specific operation: The device renders the received 3D model data using graphics libraries such as WebGL or Three.js, and displays the virtual space on the browser.

[2314] Input: 3D model data

[2315] Output: Display of virtual space

[2316] Step 3:

[2317] Users move around the virtual space and view the exhibits.

[2318] Specific actions: Users navigate the virtual space using a keyboard, mouse, or touchscreen, and can click on specific exhibits to view more information.

[2319] Input: User actions

[2320] Output:Detailed information about the exhibit

[2321] Interactive tour and sentiment analysis processing

[2322] Step 1:

[2323] Users enter questions and comments about the artwork via text on their device.

[2324] Specific behavior: The user enters a question or comment in the text box and clicks the submit button.

[2325] Input: Text data (questions and comments)

[2326] Output: Sending text data

[2327] Step 2:

[2328] The server analyzes the content of the speech using natural language processing algorithms.

[2329] Specific operation: The server receives the text data, analyzes it using a natural language processing engine (Google NLP API, SpaCy, NLTK), and infers the user's emotions and intentions.

[2330] Input: Text data

[2331] Output: Analysis results (emotions and intentions)

[2332] Step 3:

[2333] The server estimates the user's emotions and intentions and generates an appropriate response.

[2334] Specific operation: The server generates an appropriate response using a generative AI model (OpenAI GPT-3) based on the analysis results.

[2335] Input: Analysis results

[2336] Output: Response text

[2337] Step 4:

[2338] The server sends the response to the terminal and displays it to the user.

[2339] Specific operation: The server formats the generated response text as an HTTP response and sends it to the terminal, which then displays the response text to the user.

[2340] Input: Response text

[2341] Output: View the response

[2342] Handling online events and workshops

[2343] Step 1:

[2344] The server manages and notifies schedules for online events and workshops.

[2345] Specific operation: The server stores the schedule of the online event in a database and notifies participating users via email or other means.

[2346] Input: Online event information

[2347] Output: Notification message

[2348] Step 2:

[2349] The user checks the event information on their device and applies to participate.

[2350] Specific operation: The user checks the event information on the device and presses the participation application button.

[2351] Input: Event participation application

[2352] Output: Send application information

[2353] Step 3:

[2354] The server manages the participant list and controls participant access at the start of the event.

[2355] Specific operation: The server manages the participant list, creates a virtual room at the start of the event, and invites participants.

[2356] Input: Participant list

[2357] Output: Virtual room URL

[2358] Step 4:

[2359] Users join events on their devices and interact with other participants through avatars.

[2360] Specific operation: A user accesses the URL of a virtual room and interacts with other participants through their avatars.

[2361] Input: Virtual room URL

[2362] Output: Interaction through avatars

[2363] (Application example 1)

[2364] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2365] Modern retail experiences require users to explore a wide variety of products without physical constraints and to receive real-time information to support purchasing decisions. However, existing online shopping systems require a lot of effort for users to obtain detailed product information and receive appropriate recommendations, and they face challenges in providing personalized responses based on the user's emotions and intent. Furthermore, there is a lack of an environment for effectively hosting new product launches and fashion events online.

[2366] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2367] In this invention, the server includes: a means for building a community using avatars in a virtual space and providing online events and workshops; a means for analyzing users' natural language utterances and estimating their emotions; a means for an AI curator to instantly respond to users' questions and provide information; a means for updating display content based on the user's movements and actions in the virtual store; a means for searching for items in the virtual store in natural language using a smartphone or head-mounted display and displaying the results; a means for making real-time suggestions for recommended items and outfits; and a means for managing and notifying the schedule of online fashion events and workshops. This allows users to freely move around the virtual store using their own avatars, efficiently explore a variety of products, and receive personalized information and recommendations in real time. Furthermore, it also becomes possible to effectively hold new product launches and fashion events online, significantly improving the user experience.

[2368] "Virtual space" refers to a digital environment in which users can operate in a virtual realm that is different from their actual physical environment.

[2369] An "avatar" is a digital character with which a user interacts in a virtual space.

[2370] A "community" is a group of users who share a common purpose or interest and interact with each other.

[2371] An "online event" is a real-time event or occasion held over the Internet.

[2372] A "workshop" is a participatory educational and training setting for learning specific skills and knowledge.

[2373] "Natural language" refers to the language used by humans on a daily basis, which needs to be converted into a form that is easy for computers to understand.

[2374] "Emotion" is a concept that represents a human mental state, such as joy, sadness, or anger.

[2375] "AI Curator" refers to a system that uses artificial intelligence to respond to user questions and provide information.

[2376] "Document" does not necessarily mean text data, but may also include drawings, explanatory text, diagrams, etc.

[2377] "Display content" is the collection of information that a user can visually see on the screen.

[2378] A "smartphone" is a portable information terminal that can use a variety of applications in addition to the functions of a mobile phone.

[2379] A "head-mounted display" is a display device worn on the head, and is primarily used to experience virtual reality and augmented reality.

[2380] "Items" are goods or merchandise that users can view or purchase within the virtual space.

[2381] "Recommendation" refers to the presentation of specific products or information based on the user's preferences and behavior.

[2382] "Coordination suggestions" refers to providing recommendations on how to effectively combine fashion items.

[2383] "Managing your schedule" refers to scheduling and coordinating specific events and tasks.

[2384] "Notification" refers to the sending of a message to a user informing them of important information or the onset of an event.

[2385] This invention is a system that allows users to use avatars in a virtual space to explore products and engage in two-way communication, such as asking questions and sending comments. Specifically, using devices such as smartphones or head-mounted displays, users can access a virtual store, search for and view items in real time, and obtain information about artworks.

[2386] System overview and basic processing

[2387] The virtual fashion boutique system includes a smartphone, a head-mounted display, and a server as its main hardware and software components. Specific software components include a natural language processing engine and a sentiment analysis engine.

[2388] User authentication and login

[2389] The server accepts system startup and provides a page for user authentication. Using a terminal, the user accesses the login screen and enters their username and password. The server receives the entered information and performs authentication by comparing it with a database. If authentication is successful, the server sends the virtual store data to the terminal and notifies it that login was successful.

[2390] Displaying and manipulating virtual space

[2391] The server sends the virtual boutique data to the device, which then renders the received data and displays the virtual space to the user, allowing the user to move around the virtual space and view the exhibited products.

[2392] Natural Language Search and Sentiment Analysis

[2393] Users input questions and opinions about fashion items via text. The device sends the input data to the server, which then analyzes the text using a natural language processing engine. The server then infers the user's emotions and intentions, generates an appropriate response, and sends it to the device.

[2394] Display of product information and AI curator response

[2395] When a user stops in front of a specific item, the server generates detailed information related to that item and displays it to the user. The AI ​​curator instantly responds to the user's questions and provides relevant product information.

[2396] Hosting online events and workshops

[2397] The server schedules new product launches and fashion events and notifies users. Users use their devices to check event information and register for participation. The server manages the participant list and controls participant access when the event starts. The device displays the event in real time and allows users to interact with other participants.

[2398] Specific examples

[2399] Example 1: User login and virtual space display

[2400] Scenario 1: A user logs into a virtual fashion boutique

[2401] The user accesses the login screen on their smartphone and enters their username and password. The server receives the user's authentication information and compares it with the user data in the database. If authentication is successful, the server sends the virtual boutique data to the device along with a "Login successful" message. The device then renders the received data and displays the virtual space to the user.

[2402] Example 2: User questions and AI curator responses

[2403] Scenario 2: User asks about a specific dress

[2404] The user moves through the virtual space and stops in front of a particular dress. The user types "What is the trend for this dress?" into a text box. The device sends the input data to the server, which analyzes the text using a natural language processing engine. Based on the analysis results, the server generates trend information for the dress and returns a response to the user. The user checks the response and deepens their knowledge about the dress.

[2405] Example 3: Participating in an online fashion event

[2406] Scenario 3: User attends an online fashion event

[2407] The server notifies users of the schedule for a new online fashion event, "Introducing the Latest Trends." Users check the event information on their devices and apply to participate. When the event starts, the server creates a virtual room and invites participants. Users access the virtual room and participate in the event while interacting with other participants.

[2408] Prompt Sentence Examples

[2409] A user logs into a fashion boutique and asks, "Tell me about the latest dress trends." An AI curator then answers the question, and the user is invited to a new fashion event.

[2410] With the above-described configuration, the present invention realizes a virtual store where a wide variety of products can be efficiently and effectively searched for without being restricted by physical movement or time.

[2411] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2412] Step 1:

[2413] The server accepts the system startup and provides a page for user authentication.

[2414] Input: User's access request

[2415] Data calculation: Display of user authentication page

[2416] Output:Login screen

[2417] Step 2:

[2418] The user accesses the login screen on the device and enters their username and password.

[2419] Input: Username, Password

[2420] Data Computation: Input Data Collection and Formatting

[2421] Output: Authentication data

[2422] Step 3:

[2423] The server receives the entered information and authenticates it by checking it against a database.

[2424] Input: Authentication data

[2425] Data calculation: database matching, authentication calculation

[2426] Output: Authentication result

[2427] Step 4:

[2428] The server sends the authentication result to the terminal, notifying it of the success or failure of the login.

[2429] Input: Authentication result

[2430] Data Calculation: Message Generation

[2431] Output: Notification of successful or failed login

[2432] Step 5:

[2433] The server sends the virtual boutique data to the terminal.

[2434] Input: Login success notification

[2435] Data calculation: data packaging, transmission processing

[2436] Output: Virtual boutique data

[2437] Step 6:

[2438] The device renders the received data and displays the virtual space to the user.

[2439] Input: Virtual Boutique Data

[2440] Data calculation: data rendering, graphics processing

[2441] Output: Virtual space display

[2442] Step 7:

[2443] Users move around the virtual space and view the exhibits.

[2444] Input: User Action

[2445] Data calculation: Position update, display update

[2446] Output: Updated display

[2447] Step 8:

[2448] The user enters a text question about a particular item.

[2449] Input: User text input

[2450] Data calculation: Collecting and structuring input data

[2451] Output: Question data

[2452] Step 9:

[2453] The terminal sends the input data to the server.

[2454] Input: Question data

[2455] Data calculation: Data transmission processing

[2456] Output: Data sent

[2457] Step 10:

[2458] The server analyzes the text using a natural language processing engine.

[2459] Input: Data sent

[2460] Data Computing: Natural Language Analysis, Text Semantic Understanding

[2461] Output: Analysis results

[2462] Step 11:

[2463] The server estimates the user's emotions and intentions, generates an appropriate response, and sends it to the device.

[2464] Input: Analysis results

[2465] Data calculation: emotion estimation, response generation

[2466] Output: Generated response data

[2467] Step 12:

[2468] When a user stops in front of a particular item, the server generates and displays detailed information related to that item.

[2469] Input: Item stop event

[2470] Data Calculations: Finding and Generating More Information

[2471] Output: Show detailed information

[2472] Step 13:

[2473] The server manages the schedule of new product launches and fashion events and notifies users.

[2474] Input: Event schedule information

[2475] Data calculation: Schedule management, notification generation

[2476] Output: Notification message

[2477] Step 14:

[2478] The user checks the event information on the device and applies to participate.

[2479] Input: Event participation application

[2480] Data processing: collection and transmission of application data

[2481] Output: Participation application data

[2482] Step 15:

[2483] The server manages the participant list and controls participant access at the start of the event.

[2484] Input: Application data

[2485] Data operations: list management, access control

[2486] Output: Participant access control

[2487] Step 16:

[2488] The device displays the event in real time and allows users to interact with other participants.

[2489] Input: Real-time event data

[2490] Data calculation: Event display and AC function provided

[2491] Output: Real-time event display, interaction function

[2492] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2493] This invention relates to a system for providing an art appreciation experience in a virtual space, and in particular to a combined emotion engine that recognizes user emotions. Specifically, it includes functions for building an avatar community and hosting online events and workshops, analyzing users' natural language utterances to estimate their emotions, providing information about artworks through an AI curator that responds to users' questions in real time, and updating the display content based on the user's movements and actions in the virtual museum.

[2494] System overview and basic processing

[2495] The virtual museum system works in conjunction with a server and terminals. The server is responsible for data management and processing, while the terminals function as the user interface. Furthermore, an emotion engine is built in, which analyzes user emotion data and provides feedback to the entire system.

[2496] User authentication and login

[2497] 1. The server accepts the system startup and provides a page for user authentication.

[2498] 2. The user accesses the login screen on the device and enters their username and password.

[2499] 3. The server receives the entered information and authenticates it by checking it against a database.

[2500] 4. The server sends the authentication result to the terminal and notifies it of the success or failure of the login.

[2501] Displaying and manipulating virtual space

[2502] 1. The server sends the virtual museum's 3D model and artwork data to the user who has successfully logged in.

[2503] 2. The device processes the received data using a rendering engine and displays the virtual space on the user's display.

[2504] 3. Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[2505] Interactive Tours and Sentiment Analysis

[2506] 1. Users stop in front of a particular piece of art and type their question or comment into a text box.

[2507] 2. The terminal sends the entered text data to the server.

[2508] 3. The server analyzes the received text data using a natural language processing engine and uses an emotion engine to infer the user's intentions and emotions.

[2509] 4. The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[2510] 5. The server generates a response and sends it to the terminal, where it displays it to the user.

[2511] Hosting online events and workshops

[2512] 1. The server notifies registered users of new online events and workshops scheduled.

[2513] 2. The device displays the received notification to the user, and the user checks the event information.

[2514] 3. If the user wishes to register, he / she sends a participation application request to the server from his / her terminal.

[2515] 4. The server accepts the participation application and registers it on the participant list.

[2516] 5. At the start of the event, the server creates a virtual room for participants and sends them an access link.

[2517] 6. Users use their devices to access the virtual room and participate in online events and workshops.

[2518] 7. Users can interact with other participants through their avatars, chat and exchange opinions.

[2519] Specific examples

[2520] Example 1: User login and sentiment engine analysis

[2521] Scenario 1: User A logs in to the virtual museum

[2522] 1. User A accesses the login screen on the device and enters their username and password.

[2523] 2. The server receives User A's authentication information and checks it against the user data in its database.

[2524] 3. If authentication is successful, the server will send the virtual museum data to the terminal along with a "Login successful" message.

[2525] 4. The device renders the received data and displays the virtual space to User A.

[2526] Example 2: User Questions and Emotion Engine Responses

[2527] Scenario 2: User B asks about a specific painting

[2528] 1. User B moves through the virtual space and stops in front of a specific painting.

[2529] 2. User B types "What is the background of this picture?" into the text box.

[2530] 3. The device sends the input data to the server, which analyzes the text using natural language processing algorithms and an emotion engine.

[2531] 4. Based on the analysis results, the server generates customized information based on the painting's background information and the user's emotions, and returns a response to User B.

[2532] 5. User B reviews the response and learns more about the painting.

[2533] Example 3: Online workshops and the use of sentiment analysis

[2534] Scenario 3: User C attends an online workshop

[2535] 1. The server notifies user C of the schedule for a new online workshop, "Understanding Monet's Water Lilies."

[2536] 2. User C checks the event information on their device and applies to participate.

[2537] 3. At the start of the event, the server creates a virtual room and invites participants.

[2538] 4. User C accesses the virtual room and participates in the workshop while interacting with other participants.

[2539] 5. The server analyzes users' comments in real time during the workshop using an emotion engine and provides interactive content that responds to participants' emotional changes.

[2540] As explained above, by combining an emotion engine that recognizes the user's emotions, the present invention can provide a more personalized art experience and deepen the user's appreciation experience, allowing the user to enjoy a deeper art experience without being restricted by physical constraints.

[2541] The processing flow will be explained below.

[2542] User login process

[2543] Step 1:

[2544] The user operates the device and accesses the login screen, enters their username and password, and clicks the "Login" button.

[2545] Step 2:

[2546] The terminal generates a request to send the entered user name and password to the server, and sends it to the server.

[2547] Step 3:

[2548] The server checks the received username and password against a database to see if the user is valid.

[2549] Step 4:

[2550] The server generates an authentication result and sends a success or failure message to the terminal.

[2551] Step 5:

[2552] The terminal receives the authentication result and displays a login success message to the user, or an error message if authentication fails.

[2553] Displaying and manipulating virtual space

[2554] Step 1:

[2555] The server prepares to send the virtual museum's 3D models and artwork data to users who have successfully logged in.

[2556] Step 2:

[2557] The server sends the virtual museum data to the terminal.

[2558] Step 3:

[2559] The device processes the virtual museum data received using a rendering engine and displays the virtual space on the user's display.

[2560] Step 4:

[2561] Users use a mouse, keyboard, VR controller, etc. to move around the virtual space and view the exhibits.

[2562] Step 5:

[2563] The device detects the user's actions and reflects the corresponding actions (movement, change of viewpoint, selection of specific objects) in the virtual space in real time.

[2564] Interactive Tours and Sentiment Analysis

[2565] Step 1:

[2566] Users stop in front of a particular piece of art and type their question or comment into a text box.

[2567] Step 2:

[2568] The terminal generates a request for transmitting the input text data to the server, and transmits the request to the server.

[2569] Step 3:

[2570] The text data received by the server is analyzed using a natural language processing engine, and the emotion engine is used to infer the user's intentions and emotions.

[2571] Step 4:

[2572] The server generates appropriate response data based on the analysis results and customizes the description of the artwork based on emotions.

[2573] Step 5:

[2574] The server sends the generated response data to the terminal.

[2575] Step 6:

[2576] The terminal displays the received response data to the user, who then checks the detailed information.

[2577] Hosting online events and workshops

[2578] Step 1:

[2579] The server generates data to notify registered users of new online event and workshop schedules.

[2580] Step 2:

[2581] The server sends the notification data to the registered user's terminal.

[2582] Step 3:

[2583] The device displays the received notification to the user, who then checks the event information.

[2584] Step 4:

[2585] When a user wishes to register for participation, the user sends a participation application request to the server from the terminal.

[2586] Step 5:

[2587] The server accepts the participation application and registers it in the participant list.

[2588] Step 6:

[2589] At the start of the event, the server creates a virtual room exclusively for participants and sends them an access link.

[2590] Step 7:

[2591] Users use their devices to access virtual rooms and participate in online events and workshops.

[2592] Step 8:

[2593] Users interact with other participants through their avatars, chatting and exchanging op...

Claims

1. A means to build a community using avatars in a virtual space and provide online events and workshops, A means for analyzing the content of a user's natural language utterances and estimating the user's emotions; A means for AI curators to instantly respond to user questions and provide information about artworks; A means for updating the displayed content based on the user's movements and actions in the virtual museum; A system including:

2. The system according to claim 1, wherein the system analyzes text data of a user's questions and impressions using a natural language processing engine and generates an appropriate response based on the estimated user's emotions.

3. 10. The system of claim 1, wherein when a user stops in front of a particular artwork, detailed information related to that artwork is displayed.

Images