Information processing device and information processing method

Abstract

An information processing device according to one embodiment comprises: a determination unit that determines whether a conversation between a first avatar operated by a user in a virtual space and a second avatar that autonomously acts includes content following a set scenario; and an output unit that, when it is determined that the conversation includes content following the scenario, outputs data for causing the second avatar to utter a sentence including the URL of a set site.

Description

Information processing device and information processing method

[0001] The present invention relates to a technique for controlling an avatar in a virtual space.

[0002] Techniques for using avatars in virtual spaces are known. For example, Patent Literature 1 discloses an invention in which a user uses multiple avatars capable of autonomous action in a virtual space to interact with and exchange information with the avatars of other users.

[0003] Japanese Patent Application Laid-Open No. 2020-101950

[0004] The invention described in Patent Document 1 is limited to exchanging information between a plurality of avatars that require user operation.

[0005] In contrast, the present invention utilizes improved avatars to provide information tailored to the user's needs.

[0006] An information processing device according to one aspect of the present disclosure includes a determination unit that determines whether a conversation between a first avatar operated by a user in a virtual space and a second avatar acting autonomously contains content that conforms to a set scenario, and an output unit that outputs data to cause the second avatar to speak a sentence that includes the URL of a set site if it is determined that the conversation contains content that conforms to the scenario.

[0007] An information processing method according to another aspect of the present disclosure includes a step in which a computer determines whether a conversation between a first avatar operated by a user in a virtual space and a second avatar acting autonomously contains content that conforms to a set scenario, and a step in which, if it is determined that the conversation contains content that conforms to the scenario, outputting data for causing the second avatar to speak a sentence that includes the URL of a set site.

[0008] According to the present invention, it is possible to provide information that meets the needs of a user by utilizing an improved avatar.

[0009] 1 is a diagram illustrating an example of a system configuration of an information processing system 1 according to an embodiment. A diagram illustrating an example of a functional configuration of the information processing system 1. A diagram illustrating an example of a hardware configuration of an information processing device 10. A flowchart illustrating an example of an operation overview in the information processing system 1. A diagram illustrating an AI avatar database 1001. A sequence chart illustrating an example of a market guide method using an AI avatar in the information processing system 1. A diagram illustrating speech processing by an AI avatar. A flowchart illustrating an example of a method of controlling a dialogue by the information processing system 1. A diagram illustrating a conversation between a user avatar and an AI avatar. A flowchart illustrating an example of a method of controlling a conversation by an AI avatar. A diagram illustrating a certain range based on an AI avatar.

[0010] 1. Configuration FIG. 1 illustrates the system configuration of an information processing system 1 according to one embodiment. In this example, the information processing system 1 (or simply referred to as the system) is a system in which AI avatars of customers placed in a virtual space on a computer act autonomously and effectively conduct marketing activities (e.g., market guides) through interaction with user avatars. In this example, in the virtual space (or metaverse) constructed on the computer, characters representing users or computers, known as "avatars," can act freely. In the present invention, avatars are broadly divided into two types: one is referred to as a user avatar (an example of a first avatar) and the other is referred to as an AI avatar (an example of a second avatar). In this example, the user avatar is an avatar operated by an end user (or simply referred to as a user). The AI ​​avatar is an avatar autonomously controlled by a computer or program implementing AI. The market guide is a guiding behavior autonomously performed by an AI avatar in response to a user's request, including, for example, customer service, sales, or research (e.g., market research).

[0011] In virtual spaces, users can use their own avatars to access various services operated by companies or corporations (examples of customers). Meanwhile, for customers who operate services, properly managing avatars representing their companies is an essential challenge. When customers interact with users in virtual spaces, utilizing avatars that are not operated by actual humans, such as non-player characters (NPCs), is useful in terms of personnel, effort, and workload. In particular, the inventors focused on large language models (LLMs), which have made dramatic advances in the field of generative AI. LLMs possess features such as language processing capabilities using natural language, analytical capabilities, and a massive database, and also have high conversational affinity with users. This means that AI avatars equipped with LLMs are expected to have a wide range of benefits for customer marketing activities by engaging in advanced interactions (e.g., conversations) with user avatars. Therefore, the present invention presents the following system for enabling a customer's AI avatar placed in a virtual space to act autonomously and conduct marketing activities toward user avatars.

[0012] The information processing system 1 includes an information processing device 10, a user terminal 20, a customer terminal 30, a customer server 40, and a large-scale language model 90. In this example, the components of the system are connected via a network 9 as shown in Fig. 1. In this example, the network 9 is a computer network such as the Internet or a mobile network.

[0013] The information processing device 10 is an information processing device or server device in the information processing system 1. In this example, the information processing device 10 is a device related to various metaverse services, for example, managing virtual spaces and avatars. In this example, the information processing device 10 manages AI avatars for customers who operate services in the virtual space. Customers can generate, register, and configure various settings for AI avatars available in the virtual space via a customer terminal 30. The information processing device 10 controls the AI ​​avatars to act autonomously within the virtual space based on conditions set by the customer. In this example, autonomous behavior refers to the autonomous (or automatic, i.e., without real-time user control) behavior of an AI avatar placed in a specific area within the virtual space, including, for example, movement within the virtual space, character-specific behavior, interaction with a user avatar, or the use of predetermined effects. The AI ​​avatars engage in various marketing activities through interactions (e.g., conversations) with user avatars in the virtual space. The information processing device 10 works in conjunction with a large-scale language model 90 to provide users with high-quality conversations using natural language by the AI ​​avatars.

[0014] The large-scale language model 90 is an AI that includes an LLM. In this example, the large-scale language model 90 generates conversation content for an AI avatar in response to a request from the information processing device 10. The large-scale language model 90 outputs sentences to be spoken by the AI ​​avatar based on prompts input from the information processing device 10. In this example, the prompts are instructions or questions that cause the LLM to generate sentences, etc. The information processing device 10 obtains sentences from the large-scale language model 90 and uses them for the AI ​​avatar's speech.

[0015] The user terminal 20 is a terminal owned and used by an end user. The user terminal 20 includes, for example, a smartphone, a tablet, or a personal computer. In this example, the user is the subject who operates a user avatar (i.e., their own avatar). The user terminal 20 can access the metaverse service of the information processing device 10. For example, the user terminal 20 accepts operations related to the user avatar from the user. In the virtual space, the user can freely control the user avatar according to their own operations. This allows the user to interact with various avatars, for example. The user terminal 20 also acquires various data from the user. The user terminal 20 acquires information related to the user's profile or values ​​(hereinafter referred to as "value information") and transmits it to the information processing device 10.

[0016] The customer terminal 30 is a terminal owned and used by a customer. The customer terminal 30 includes, for example, a smartphone, a tablet, or a personal computer. The customer includes, for example, a company, a corporation, an organization, or a local government. In this example, the customer terminal 30 cooperates with the information processing device 10 to generate, register, and manage an AI avatar in response to a customer's request. The customer terminal 30 can acquire settings, conditions, and various parameters related to the AI ​​avatar from the customer, and can provide the customer with information about the AI ​​avatar's history or user avatars with which the customer has interacted.

[0017] The customer server 40 is a server device that manages a site owned or operated by the customer. In this example, the site is a dedicated service on the Internet that the customer manages independently, and includes, for example, a website, a homepage, or a social networking site. In the information processing system 1, the AI ​​avatar presents, as part of marketing activities, for example, a means of accessing the customer site to the user avatar (or the user himself / herself) as an external link. The external link includes, for example, a URL.

[0018] A user can access the customer site from various access means using the user terminal 20. The customer server 40 provides services to the user via the user terminal 20.

[0019] 2 is a diagram illustrating an example of the functional configuration of the information processing system 1. In this embodiment, the information processing device 10 has functional blocks (or components) including an acquisition unit 11, a determination unit 12, a generation unit 13, an output unit 14, a conversation control unit 15, a storage unit 191, and a control unit 192. In this example, the storage unit 191 stores, for example, various data including a database, programs, and software modules. In this example, the control unit 192 performs various controls.

[0020] The acquisition unit 11 acquires various pieces of information in the virtual space. The acquisition unit 11 acquires information about the actions of the first avatar and the second avatar. In this example, the acquisition unit 11 acquires information about a conversation between the first avatar and the second avatar.

[0021] The determination unit 12 determines whether a conversation between a first avatar operated by a user in a virtual space and a second avatar acting autonomously includes content that conforms to a set scenario. In this example, the scenario is a plot (or action definition) related to marketing activities that is implemented in advance in the second avatar according to the customer's objectives. The scenario includes, for example, recommendations, product presentations, or service introductions. The determination unit 12 can determine whether the conversation conforms to the scenario based on the statements of the first avatar, etc. The determination unit 12 cooperates with the generation unit 13 and controls the second avatar according to the determination result.

[0022] When the generation unit 13 determines that the conversation contains content that conforms to the scenario, it uses an LLM (e.g., a large-scale language model 90) to generate sentences that conform to the scenario. Here, the information processing device 10 and the large-scale language model 90 are linked together via, for example, a Retrieval-Augmented Generation (RAG) 110. In this example, RAG is a prompt extension technology that implements a search function that responds to various requests in order to improve the accuracy of responses from the LLM. Prompts to be input into the large-scale language model 90 are automatically generated, for example, by the RAG 110.

[0023] The RAG 110 searches for various information from a database (an example of the storage unit 191). In this example, the RAG 110 can select and acquire candidates from a list (an example of a list) of recommended products, services, or sites that fit the scenario.

[0024] The generation unit 13 cooperates with the RAG 110 to acquire information from a database. The generation unit 13 (or the RAG 110) cooperates with the large-scale language model 90 to generate sentences. In this example, the generation unit 13 generates sentences including the URL of a site selected from a pre-set list in response to a utterance from the first avatar. In another example, the generation unit 13 extracts user needs from the utterance of the first avatar and selects a site URL from the list by keyword search based on the user needs. Note that the user needs include the user's internal demands, requests, or desires.

[0025] When it is determined that the conversation includes content that conforms to the scenario, the output unit 14 outputs data (hereinafter referred to as "utterance data") for causing the second avatar to utter a sentence including the URL of the set site. The output unit 14 cooperates with the generation unit 13 and the large-scale language model 90 to acquire the utterance data. In this example, the output unit 14 outputs data for causing the second avatar to utter the sentence generated by the generation unit 13. In another example, the output unit 14 causes the second avatar to utter the sentence including the URL as the final sentence, and outputs data (e.g., a questionnaire) for concluding the conversation.

[0026] The conversation control unit 15 performs various controls on the behavior (e.g., conversation) of the second avatar in the virtual space. The conversation control unit 15 starts a conversation when the first avatar enters a certain range based on the second avatar. In this example, the certain range is a specific area in the virtual space, and includes, for example, a territory, object, or personal space associated with the second avatar. The certain range includes an area having a virtual (or physical) shape. For example, the certain range has a shape in which the front of the second avatar is wider than the back. In the virtual space, it is possible to implement a situation in which the first avatar and the second avatar face each other and have a conversation, and the certain range is determined according to such specifications.

[0027] Additionally, the conversation control unit 15 resets the conversation when the first avatar moves out of a certain range. In this example, the conversation control unit 15 can perform various controls on the second avatar, such as resetting the conversation, taking over the conversation, or restricting the actions of the second avatar, in order to avoid a loss of user satisfaction or a degradation of the user experience.

[0028] FIG. 3 is a diagram illustrating an example of the hardware configuration of the information processing device 10. The information processing device 10 is physically configured as a computer including a processor 101, a memory 102, a storage 103, a communication device 104, an input device (optional), a display device (optional), and a bus connecting these. Each of these devices operates using power supplied from a battery (not shown). In the following description, the term "device" can be interpreted as a circuit, device, unit, etc. The hardware configuration of the information processing device 10 may be configured to include one or more of the devices shown in FIG. 3, or may be configured without including some of the devices. Furthermore, the information processing device 10 may be configured by communicating and connecting multiple devices each having a different housing.

[0029] Each function of the information processing device 10 is realized by loading specified software (programs) onto hardware such as the processor 101, memory 102, etc., so that the processor 101 performs calculations, controls communication via the communication device 104, and controls at least one of reading and writing data in the memory 102 and storage 103.

[0030] The processor 101 controls the entire computer by running, for example, an operating system. The processor 101 may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. Furthermore, for example, a baseband signal processing unit, a call processing unit, etc. may be realized by the processor 101.

[0031] The processor 101 reads programs (program codes), software modules, data, etc. from at least one of the storage 103 and the communication device 104 into the memory 102 and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described below. The functional blocks of the information processing device 10 may be implemented by a control program stored in the memory 102 and running on the processor 101. Various processes may be executed by one processor 101, or may be executed simultaneously or sequentially by two or more processors 101. The processor 101 may be implemented by one or more chips. The programs may be transmitted to the information processing device 10 via a telecommunications line.

[0032] The memory 102 is a computer-readable recording medium and may be configured by at least one of, for example, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 102 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 102 can store executable programs (program codes), software modules, etc. for implementing the method according to this embodiment.

[0033] Storage 103 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 103 may also be called an auxiliary storage device.

[0034] The communication device 104 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module.

[0035] Each device, such as the processor 101 and the memory 102, is connected by a bus for communicating information. The bus may be configured using a single bus, or different buses may be used between each device.

[0036] The information processing device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 101 may be implemented using at least one of these pieces of hardware.

[0037] In this example, the programs stored in the storage 103 include a program (hereinafter referred to as a "server program") for causing a computer to function as a server in the information processing system 1. When the processor 101 is executing the server program, the processor 101, the memory 102, the storage 103, and the communication device 104 are examples of functional blocks for operating the information processing device 10. The processor 101 is an example of an acquisition unit 11, a determination unit 12, a generation unit 13, an output unit 14, a conversation control unit 15, and a control unit 192. At least one of the memory 102 and the storage 103 is an example of a storage unit 191. The communication device 104 is an example of the acquisition unit 11 and the output unit 14.

[0038] The server program includes a program module for autonomously controlling an AI avatar (hereinafter referred to as the "avatar control module"). In detail, the functions of this avatar control module are an acquisition unit 11, a determination unit 12, a generation unit 13, an output unit 14, and a conversation control unit 15.

[0039] Although detailed description will be omitted, the user terminal 20 and the customer terminal 30 are computers having a processor, memory, storage, communication device, display device, input device, and output device, specifically, for example, a smartphone, tablet terminal, or personal computer. In this example, the programs stored in the storage of the user terminal 20 and the customer terminal 30 include a program (hereinafter referred to as a "client program") for causing the computer to function as a client in the information processing system 1.

[0040] The customer server 40 is a computer having a processor, memory, storage, and a communication device. In this example, the programs stored in the storage of the customer server 40 include a program (hereinafter referred to as a "customer server program") for causing the computer to function as a customer server in the information processing system 1. The configuration of the information processing system 1 has been described above. Next, the operation of the information processing system 1 will be described.

[0041] 2. Operation Figure 4 is a flowchart illustrating an example of an outline of the operation of the information processing system 1. The following operation represents an outline of the processing executed by the information processing system 1 as a whole, from the customer setting up an AI avatar to the marketing use of the AI ​​avatar. In the following example, the processing in the information processing device 10 is performed by the avatar control module.

[0042] In step S1, the information processing device 10 cooperates with the customer terminal 30 and accepts input of initial settings for the AI ​​avatar from the customer. In this example, the customer generates a 3D model related to the AI ​​avatar, sets a scenario, and registers a link list via the customer terminal 30. The information processing device 10 acquires various data related to the AI ​​avatar from the customer terminal 30 and records it in a database. Here, the database for managing the AI ​​avatar will be described.

[0043] FIG. 5 is a diagram illustrating an example of an AI avatar database 1001. In this example, the AI ​​avatar database 1001 includes multiple records related to AI avatars. Each record corresponds to information for each AI avatar (or for each customer). Each record includes an avatar ID, an avatar name, setting information, a scenario, link information, and placement information. The avatar ID is identification information unique to each AI avatar. The avatar name is a name for identifying the AI ​​avatar. The setting information is information related to the settings of the AI ​​avatar. Here, data related to attributes such as the appearance or inner self of the AI ​​avatar is recorded. For example, a 3D model representing the appearance of the AI ​​avatar is recorded in the database. Alternatively, the personality representing the inner self of the AI ​​avatar is recorded in the database. The inner self (or personality) record may include parameters for defining behavior (e.g., statements) according to the personality of the AI ​​avatar. The scenario is a scenario related to response control of the AI ​​avatar and includes, for example, a requirements definition, a standard Q&A format (so-called Q&A information), conditions, or parameters. The link information is a list of links to external sites operated by the customer. In this example, the database defines a correspondence between scenarios and presented links (e.g., URLs). The placement information is information regarding the placement and behavior of an AI avatar in a virtual space, including, for example, the location of an area in the virtual space where the AI ​​avatar is placed and the behavior of the AI ​​avatar in that area. The information processing device 10 can use the AI ​​avatar database 1001 to register AI avatars, control them in the virtual space, and use them for searches. The AI ​​avatar database 1001 manages the AI ​​avatars for each customer by adding customer identification information (e.g., customer ID) to the AI ​​avatar.

[0044] The information processing device 10 may also manage information about users and user avatars in a database or the like. For example, the information processing device 10 acquires value information indicating the user's values ​​from the user terminal 20 in advance and manages the value information by linking it to the user avatar. The value information is used when the AI ​​avatar interacts with the user. The use of the value information will be described later.

[0045] Returning to FIG. 4 , in step S2, the information processing device 10 controls the autonomous behavior of the AI ​​avatar in the virtual space. The information processing device 10 reads the AI ​​avatar database 1001 and places the AI ​​avatar in a specific area in the virtual space. The information processing device 10 causes the AI ​​avatar to act autonomously in accordance with the settings in the database. This allows the AI ​​avatar to act autonomously in the virtual space and perform marketing activities based on a scenario. Essentially, the AI ​​avatar is set to perform market guidance in response to actions from a user avatar. Therefore, it is desirable for the autonomous behavior of the AI ​​avatar to be behavior that is easily contacted by the user avatar. The following processing is an example of operation when a user avatar and an AI avatar interact with each other.

[0046] In step S3, the information processing device 10 causes the AI ​​avatar to converse with the user avatar. This process is performed in cooperation with the information processing device 10 and the large-scale language model 90. For example, the information processing device 10 determines whether the conversation is progressing according to a scenario, and the large-scale language model 90 generates sentences to be spoken by the AI ​​avatar. Finally, the information processing device 10 presents the user with the URL of the customer site recorded in the AI ​​avatar database 1001. The detailed process in this step will be described later.

[0047] In step S4, the information processing device 10 collects the content of the conversation between the AI ​​avatar and the user. The behavioral history, conversation history, and market guide history of the AI ​​avatar are recorded in various databases and collected for each customer. The information processing device 10 outputs various information to the customer terminal 30. The customer obtains various information from the information processing device 10 via the customer terminal 30. This information is used for marketing analysis, user trend analysis, or policy planning.

[0048] This concludes the description of the outline of the operations of the information processing system 1. Note that the conversation between the AI ​​avatar and the user avatar in step S3 is carried out using LLM, and therefore detailed processing will be exemplified in Section 2-1 or 2-2.

[0049] 2-1. Market guide method using an AI avatar Figure 6 is a sequence chart illustrating a market guide method using an AI avatar in the information processing system 1. Here, an example of the operation when a user avatar talks to an AI avatar in a virtual space is described. Note that the processing in the information processing device 10 in this sequence is performed by the avatar control module. The avatar control module includes various functional elements, a database, and the RAG 110.

[0050] In step S101, the information processing device 10 acquires a conversation with a user from the user terminal 20. In this example, in the virtual space, a user avatar interacts with an AI avatar using various communication means (e.g., text or chat). More specifically, data is transmitted via an API (Application Programming Interface) or the like from a front end where the user can directly interfere or operate via the user terminal 20 to a back end of the information processing device 10, which is the main processing subject.

[0051] In step S102, the information processing device 10 determines whether the conversation between the user avatar and the AI ​​avatar includes content that conforms to the set scenario. For example, if the scenario is a guide to recommended products (or services), the information processing device 10 determines whether the user's utterance includes a request for a recommendation. The determination is made based on words or keywords included in the user's utterance.

[0052] Here, if the information processing device 10 determines that the conversation contains content that conforms to the scenario, the information processing device 10 extracts user needs from the content of the user's utterance. The information processing device 10 cooperates with the RAG 110 and searches a database (e.g., keyword search) based on the extracted user needs. The RAG 110 references the AI ​​avatar database 1001 and acquires data according to the user needs. The data according to the user needs includes, for example, information about products, services, or sites handled by the customer. On the other hand, if the information processing device 10 determines that the conversation does not contain content that conforms to the scenario, the information processing device 10 does not need to search the database.

[0053] In step S103, the information processing device 10 outputs a request to the large-scale language model 90 to generate a sentence to be spoken by the AI ​​avatar. The request to the large-scale language model 90 is made via various prompts automatically generated by the RAG 110. The large-scale language model 90 generates the sentence based on the input prompt. The prompt includes, for example, the content of the conversation (e.g., the most recent statement made by the user avatar), search results, and specifications related to the sentence (e.g., specifications regarding the writing style or tone).

[0054] In step S104, the information processing device 10 acquires the utterance content from the large-scale language model 90. The information processing device 10 performs post-processing on the acquired utterance content as needed. For example, if user needs are extracted in step S102, the post-processing may include adding link information (e.g., a URL) to the utterance content. Other post-processing may include editing text (or images, etc.), correcting sentences, and adding data.

[0055] In step S105, the information processing device 10 outputs speech data. Data is output from the back end of the information processing device 10, and the data is reflected as speech by the AI ​​avatar at the front end on the metaverse service side. The user terminal 20 acquires the speech content of the AI ​​avatar as a response to the user avatar. The user can check the speech data via the user terminal 20. Here, the overall flow of the process will be described.

[0056] FIG. 7 is a diagram illustrating speech processing by an AI avatar. FIG. 7 is an example of a schematic diagram (or processing model) showing a series of processes from step S101 to step S105. In this example, schematic diagram 1002 shows a method for generating sentences to be spoken by an AI avatar depending on whether or not a scenario exists for the conversation content. Note that the information processing device 10 requests the large-scale language model 90 to generate sentences, regardless of whether or not the conversation includes content that conforms to a scenario. This allows the information processing device 10 to obtain high-quality sentences that allow the AI ​​avatar to converse more naturally with the user.

[0057] Returning to FIG. 6, in step S106, the information processing device 10 records the conversation history between the user avatar and the AI ​​avatar in a database. This allows the information processing device 10 to accumulate records of marketing activities using the AI ​​avatar in the database. The information processing device 10 outputs various data in response to requests from customers via the customer terminal 30. Specific examples of conversations between a user avatar and an AI avatar are given in Section 2-2.

[0058] As described above, the information processing device 10 can use the LLM to provide a market guide using an AI avatar. The information processing device 10 can provide various information to the user based on the scenario determination. The information processing device 10 can provide customers with information that can be used for marketing. Next, a method for controlling dialogue by the information processing system 1 will be described.

[0059] 2-2. Dialogue Control Method by Information Processing System 1 FIG. 8 is a flowchart illustrating a dialogue control method by the information processing system 1. Here, a control method for dialogue flow will be described along with an actual chat-style conversation between a user avatar and an AI avatar. The processing in the information processing device 10 in this flow is performed by an avatar control module. In step S21, the information processing device 10 controls the AI ​​avatar to listen to the user's needs. An example of the conversation will now be described.

[0060] FIG. 9 illustrates a conversation between a user avatar and an AI avatar. In this example, conversation 1003 is the content of a text-based conversation in a so-called chat format. Here, display screen D1 displayed on the user terminal 20 is shown as an example. Icon I1 is the icon of a user avatar. Icon I2 is the icon of an AI avatar. Message M represents the content of messages sent by the user avatar and the AI ​​avatar. Messages M11 to M14 represent messages sent by the user avatar, and messages M21 to M24 represent messages sent by the AI ​​avatar. This example assumes a case in which the user avatar speaks to the AI ​​avatar (e.g., when message M11 is sent). In response, as indicated by the sentence in message M21, "Hello! We'll help you find the perfect product for you! Is there anything you're looking for right now?", the AI ​​avatar collects information to extract the user's needs. The text represented by messages M21 to M24 is generated by the information processing device 10 using the large-scale language model 90.

[0061] Returning to FIG. 8 , in step S22, the information processing device 10 determines whether the need requirements are satisfied from the user's response. If the need requirements are satisfied (step S22: YES), the information processing device 10 proceeds to step S25. On the other hand, if the need requirements are not satisfied (step S22: NO), the information processing device 10 proceeds to step S23. In step S23, the information processing device 10 controls the AI ​​avatar to explore the user's reaction. Furthermore, in step S24, the information processing device 10 extracts a rough outline of the user's needs from the content of the user's conversation. Note that the processes of steps S23 to S24 are repeated until the need requirements in step S22 are satisfied.

[0062] Referring again to Figure 9, in response to the user's reply in message M12, "I'm looking for something sweet and delicious," the AI ​​avatar responds in message M22, "I see! Do you have any requests for your favorite sweets or favorite shops? For example, how about chocolate, cake, or ice cream?" to find out the user's reaction or request (corresponding to step S23), and extracts general needs, such as candidate sweets (corresponding to step S24).

[0063] Returning to Fig. 8, if the needs requirements are satisfied (step S22: YES), the information processing device 10 proceeds to step S25. In step S25, the information processing device 10 identifies the details of the user needs.

[0064] Referring back to FIG. 9 , the information processing device 10 determines that the need requirement is satisfied based on the content of message M13, "Hmm, I guess I like Basque cheesecake. I like something light and not too cheese-heavy." In this example, the need requirement is determined based on a scenario. For example, if the scenario includes a recommended product, the need requirement is set to a specific product name, such as "Basque cheesecake." In response, the AI ​​avatar presents details of the user's need in message M23, saying, "I understand! You're looking for a Basque cheesecake with a mild cheese flavor. If there are any similar sweets, would you like that too?" At this time, the AI ​​avatar obtains permission or confirmation from the user to provide information. Alternatively, the AI ​​avatar may respond with additional requests, etc.

[0065] Returning to Fig. 8, in step S26, the information processing device 10 provides information according to the user's needs, including information about products or sites.

[0066] Referring again to FIG. 9 , as indicated by the sentence in message M24, "I was looking for a product that met my needs: (1) Basque cheesecake (2) a mild cheese flavor. I'm sharing a page that seems relevant, so please take a look if you'd like. 1. https: / / ... 2....," the AI ​​avatar presents site information (e.g., URL) for a product that meets the user's needs. In this example, the site information is the URL of a site preset in the database. The information processing device 10 references the AI ​​avatar database 1001 and selects the URL of a site that meets the user's needs from a list of link information.

[0067] Returning to FIG. 8 , in step S27, the information processing device 10 presents a questionnaire to the user via the AI ​​avatar. This processing is intended to be used for customer marketing activities and to improve the conversation quality of the AI ​​avatar. Note that the information processing device 10 may end the conversation with the user when the AI ​​avatar presents the site URL or the questionnaire.

[0068] As described above, the information processing system 1 can provide the user with valuable information tailored to the user's needs by utilizing an AI avatar. Furthermore, by using the large-scale language model 90, the information processing system 1 can acquire high-quality (or natural) sentences for the AI ​​avatar to speak. So far, we have explained how the information processing system 1 operates to control dialogue. Next, we will explain how an AI avatar acting autonomously in a virtual space controls a conversation.

[0069] 2-3. Method for Controlling Conversation by an AI Avatar FIG. 10 is a flowchart illustrating a method for controlling a conversation by an AI avatar. Here, we will explain a method for controlling an AI avatar in a virtual space until a conversation between a user avatar and an AI avatar begins. The processing in this flow in information processing device 10 is performed by an avatar control module.

[0070] In step S31, the information processing device 10 controls the AI ​​avatar to behave autonomously. Autonomous behavior refers to, for example, an AI avatar placed in a specific area in the virtual space wandering around at will or using various actions or effects. These actions are controlled by the information processing device 10. This allows the AI ​​avatar to be operated without direct intervention (e.g., operation) by a customer's employees or the like.

[0071] In step S32, the information processing device 10 detects a user avatar that has entered a certain range based on the AI ​​avatar. Here, the certain range will be described.

[0072] FIG. 11 is a diagram illustrating a certain range based on an AI avatar. In this example, display screen D2 is an example of a screen displayed on user terminal 20. That is, display screen D2 displays an AI avatar visible to the user (user avatar). User avatar UA1 is a 3D model representing the user avatar. AI avatar AI10 is a 3D model representing the AI ​​avatar. In this figure, the avatar is depicted as a silhouette. Therefore, the three-dimensional appearance of the avatar, the perspective of the object, and the depth of the virtual space in the figure are not necessarily accurate. Range R10 is a display example of a certain range based on the AI ​​avatar. Range R10 is an object (ring-shaped in the figure) that surrounds the AI ​​avatar in the virtual space. When the AI ​​avatar's feet (e.g., a certain point) are used as the reference point, range R10 is a range that includes the area inside a line bounded by points at a specific distance from that point (or a range that includes that line).

[0073] Here, the function of the range R10 will be described. The range R10 has a function for detecting the approach of a user avatar. Alternatively, in addition to the above detection function, the range R10 also functions as a marker for the user avatar. In this example, the information processing device 10 changes the display color of the range R10 according to various conditions. For example, the information processing device 10 controls the ring color to yellow when the user avatar is outside the range R10, and changes the ring color to green when intrusion into the range R10 is detected. The color variation of the ring is used to manipulate the impression on the user (e.g., yellow represents normality, green represents gentleness, etc.). The information processing device 10 can change the marker for the range R10 according to the user avatar's approach to the AI ​​avatar (e.g., whether it has entered a certain range). This has the effect of making it easier for the user avatar to talk to the AI ​​avatar (i.e., increasing conversation opportunities). Note that the range R10 may have a shape that is wider in front of the AI ​​avatar than behind it, making it easier for the user avatar to talk to the AI ​​avatar. Alternatively, the information processing device 10 may provide the range R10 with a reset function for resetting the conversation when the user avatar leaves the range R10. This allows the information processing device 10 to perform various controls such as resetting the conversation, taking over the conversation, or restricting actions, thereby preventing a loss of user satisfaction or a degradation of the user experience.

[0074] Returning to Fig. 10, in step S33, the information processing device 10 determines whether a conversation start condition is satisfied. In this example, the start condition is, for example, a conversation request by the user avatar. The following description will be given assuming that the user avatar is talking to the AI ​​avatar.

[0075] If the conversation start condition is satisfied (step S33: YES), the information processing device 10 proceeds to step S34. On the other hand, if the conversation start condition is not satisfied (step S33: NO), the information processing device 10 returns to step S31. In this example, the processes of steps S31 to S32 are repeated until the conversation start condition in step S33 is satisfied, that is, until the AI ​​avatar speaks to the user avatar. Note that the entry of the user avatar into a certain range of the AI ​​avatar may be set in a database or the like as an opportunity to start a conversation.

[0076] In step S34, the information processing device 10 causes the AI ​​avatar to start a conversation with the user. In this example, a chat screen such as that described in Fig. 9 is displayed on the user terminal 20. The information processing device 10 starts a market guide using the AI ​​avatar in accordance with the user needs.

[0077] Although the conditions for starting a conversation have been described above, the information processing device 10 may similarly control the end of the conversation. When a condition for ending a conversation is met, the information processing device 10 controls the conversation with the user avatar to be stopped. Examples of the condition for ending a conversation include the user avatar going out of range R10 during the conversation, or the user avatar facing the opposite direction from the AI ​​avatar.

[0078] As a result, the information processing system 1 can control the autonomous behavior of an AI avatar in a virtual space. The information processing system 1 can control various conversations with the AI ​​avatar. This increases opportunities for interaction with users and enables more effective marketing activities.

[0079] 3. Modifications The present invention is not limited to the above-described embodiment, and various modifications are possible. Some modifications will be described below. Two or more of the following features may be combined and applied.

[0080] (1) Information Processing System 1 The hardware configuration and network configuration of the information processing system 1 are not limited to those exemplified in the embodiment. The information processing system 1 may have any hardware configuration and network configuration as long as the required functions can be realized. For example, multiple physical devices may cooperate to function as the information processing system 1. For example, at least a portion of the information processing device 10 may be implemented in a large-scale language model 90. For example, the large-scale language model 90 may have at least a portion of the functions of the RAG 110, among the functions related to the information processing device 10.

[0081] (2) Information Processing Device 10 Some of the functions of the information processing device 10 may be implemented in another server. This server may be, for example, a physical server or a virtual server (including a so-called cloud). Furthermore, the correspondence between functional elements and hardware is not limited to that illustrated in the embodiment. For example, in the embodiment, at least some of the functions described as being implemented in the information processing device 10 may be implemented in another device or system. Conversely, at least some of the functions described as being implemented in another device or system may be implemented in the information processing device 10. In this example, at least some of the information processing device 10 may be implemented in the customer terminal 30 or the customer server 40. For example, the customer terminal 30 may have at least some of the functions of the RAG 110, among those related to the information processing device 10. The customer terminal 30 may cooperate with the RAG 110 to search the behavioral history of an AI avatar.

[0082] (3) User Terminal 20, Customer Terminal 30, and Customer Server 40 The user terminal 20, customer terminal 30, and customer server 40 are not limited to those illustrated in the embodiment. The user terminal 20, customer terminal 30, and customer server 40 may perform the above-described processes using any display screen, input device, external device, or various UIs. The user terminal 20 may be equipped with any function necessary for the user to operate an avatar. In this example, the user terminal 20 may be equipped with a UI for operating a user avatar or a function for conversing with an AI avatar (e.g., chat, direct mail (DM), voice conversation function, etc.). The customer terminal 30 may be equipped with any function necessary for the customer to execute processes related to the AI ​​avatar. In this example, the customer terminal 30 may have a function for generating and managing link information. The customer server 40 may be equipped with any function necessary for realizing functions related to the customer's purpose or user usage. In this example, the customer server 40 may be equipped with functions such as posting product information, issuing coupons, and providing various site services.

[0083] (4) Large-Scale Language Model 90 The large-scale language model 90 is not limited to the example illustrated in the embodiment. The large-scale language model 90 may implement any machine learning model as long as it can realize the required functions. At least a part of the functions of the large-scale language model 90 may be implemented in the information processing device 10. The large-scale language model 90 may accept prompt input in any data format. The large-scale language model 90 may output data in any format. For example, this data may be text-based, CSV (comma separated values), or table format. The large-scale language model 90 may directly cause an AI avatar to speak using the generated sentences.

[0084] (5) Operation Overview: The flowchart in FIG. 4 merely illustrates an example of the operation, and the operation of the information processing system 1 is not limited to this. Some of the illustrated operations may be modified or omitted, the order may be changed, or new operations may be added. In step S2, the information processing device 10 may control the AI ​​avatar to speak to the user on its own. In this case, the information processing device 10 may cause the AI ​​avatar to perform promotional activities aimed at increasing customer awareness of the service. Note that, to avoid a degradation of the user experience, the information processing device 10 may control the above promotional activities only for users who have never conversed with the AI ​​avatar. Furthermore, in step S2, the information processing device 10 may control the behavior of the AI ​​avatar based on user value information. The information processing device 10 may estimate a match rate with the target AI avatar based on the user's preferences, hobbies, tastes, needs, etc. The information processing device 10 may control the behavior of the AI ​​avatar based on the match rate with the user's values. For example, the information processing device 10 may control the AI ​​avatar to have more opportunities to interact with the user if the AI ​​avatar has a higher match rate with the user, or to provide a simpler market guide if the AI ​​avatar has a lower match rate. In step S3, the information processing device 10 may use any communication format to facilitate a conversation between the AI ​​avatar and the user avatar, such as text, voice, or action. Alternatively, the information processing device 10 may cause the AI ​​avatar to have a conversation that is tailored to the user's hobbies and preferences based on value information.

[0085] (6) Market Guide Method Using an AI Avatar The sequence chart shown in FIG. 6 merely illustrates an example of the operation, and the operation of the information processing system 1 is not limited to this. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In step S102, the information processing device 10 may use any method to determine the scenario. In addition to (or instead of) the process of extracting needs from a conversation with a user, which has been described as an example, the information processing device 10 may perform a determination process on a scenario set based on, for example, the number of interactions between the AI ​​avatar and the user avatar. In step S102, when the information processing device 10 determines that the conversation contains content in accordance with the scenario, the RAG 110 may search for any data. The RAG 110 may search for data related to the user (or the user avatar). The RAG 110 may acquire data corresponding to the user's values. This makes it easier for the information processing device 10 to present information that meets the user's needs. Alternatively, regardless of whether the conversation includes content according to a scenario, the RAG 110 may acquire data related to the user avatar or the metaverse to facilitate the conversation with the user. For example, the RAG 110 may acquire value information of the user. In step S103, the RAG 110 may generate any prompt. For example, the prompt may include a conversation history, a scenario, link information, or value information.

[0086] (7) Dialogue Control Method by Information Processing System 1 The flowchart shown in FIG. 8 merely illustrates one example of the operation, and the operation of the information processing system 1 is not limited to this. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In step S22, the needs requirements may be set in any manner. Furthermore, any method may be used to determine whether the needs requirements are met. In step S23, the information processing device 10 may introduce recommended products or services and explore the user's reaction. In this case, the recommendations may be obtained from a database based on the user's conversation content, value information, or the like.

[0087] (8) Method for Controlling Conversation by an AI Avatar The flowchart shown in FIG. 10 merely illustrates an example of the operation, and the operation of the information processing system 1 is not limited to this. Some of the illustrated operations may be modified or omitted, the order may be changed, or new operations may be added. In step S32, the information processing device 10 may control the detection range of the AI ​​avatar based on the user's value information. For example, the information processing device 10 may control the detection range of the AI ​​avatar based on the user's value information. For example, the information processing device 10 may control the detection range of the AI ​​avatar based on the user's preferred appearance, such that the AI ​​avatar is more easily detected, or the AI ​​avatar is kept near the user avatar for a longer period of time. In step S33, the start condition for the conversation (or scenario) may be any condition. For example, the start condition may be based on a user operation triggered by a match rate according to the value information exceeding a threshold. Note that while the display screen D2 illustrates a third-person perspective (also referred to as a third-person perspective) based on the user avatar, the display on the user terminal 20 may be any type, such as a first-person perspective, a bird's-eye view, or a display using VR (Virtual Reality).

[0088] (9) Database (Data) The database (or the data itself) of the information processing system 1 shown in FIGS. 5, 9, and 11 is not limited to the example shown in the embodiment. In this example, any type of data may be registered in the database. The AI ​​avatar database 1001 may record any type of data. For example, tags, keywords, labels, etc. may be added to scenario or link information. The information processing device 10 may also manage information about user avatars. In this case, the information processing device 10 may use the information about the user avatar when controlling the AI ​​avatar. The data recorded in the database may be in any format, such as text, image, video, or audio. The conversation 1003 may be recorded in the AI ​​avatar database 1001. For example, records of various conversations may be accumulated in the database in association with the behavioral history of the AI ​​avatar.

[0089] (10) The various other programs executed by the processor 101 may be provided by downloading via a network such as the Internet, or may be provided in a state recorded on a computer-readable non-transitory recording medium such as a DVD-ROM. Each processor may be, for example, a CPU, an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit).

[0090] The block diagrams used to explain the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of hardware and / or software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are directly or indirectly connected (e.g., wired, wireless, etc.) and these multiple devices. The functional block may also be realized by combining software with the single device or multiple devices.

[0091] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs transmission is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on how these functions are implemented.

[0092] For example, the information processing device 10 according to an embodiment of the present disclosure may function as a computer that performs the processing of the present disclosure.

[0093] Each aspect or embodiment described in the present disclosure may be applied to at least one of systems using LTE (Long Term Evolution), LTE-Advanced (LTE-A), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (New Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-Wide Band), Bluetooth (registered trademark), or other suitable systems, and next-generation systems enhanced based on these. Furthermore, a combination of multiple systems (e.g., a combination of at least one of LTE and LTE-A with 5G) may also be applied.

[0094] The order of the procedures, sequences, flowcharts, etc. of each aspect or embodiment described in this disclosure may be changed unless it is inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0095] Input and output information may be stored in a specific location (for example, memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be sent to another device.

[0096] The determination may be made based on a value represented by one bit (0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0097] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0098] Software, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise, should be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc. Additionally, software, instructions, information, etc. may be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then such wired and / or wireless technologies are included within the definition of a transmission medium.

[0099] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof. Note that terms described in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings.

[0100] Furthermore, the information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values ​​from a predetermined value, or may be expressed using other corresponding information.

[0101] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0102] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must in some way precede the second element.

[0103] The "unit" in the configuration of each of the above devices may be replaced with "means," "circuit," "device," or the like.

[0104] When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, when the term "or" is used in this disclosure, it is not intended to be an exclusive or.

[0105] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0106] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different."

[0107] 1...information processing system, 10...information processing device, 20...user terminal, 30...customer terminal, 40...customer server, 90...large-scale language model, 9...network, 11...acquisition unit, 12...determination unit, 13...generation unit, 14...output unit, 15...conversation control unit, 191...storage unit, 192...control unit, 110...RAG, 101...processor, 102...memory, 103...storage, 104...communication device, 1001...AI avatar database, 1002...schematic diagram, 1003...conversation, AI...AI avatar, UA...user avatar, D...display screen, I...icon, M...message, R...range

Claims

1. An information processing device having: a judgment unit that judges whether a conversation between a first avatar operated by a user in a virtual space and a second avatar that acts autonomously contains content that conforms to a set scenario; and an output unit that outputs data to cause the second avatar to speak a sentence that includes the URL of a set site if it is determined that the conversation contains content that conforms to the scenario.

2. An information processing device as described in claim 1, further comprising a generation unit that generates a sentence according to the scenario using an LLM when it is determined that the conversation contains content that conforms to the scenario, and wherein the output unit outputs data for causing the second avatar to speak the sentence generated by the generation unit.

3. The information processing device according to claim 2, wherein the generating unit generates a sentence including a URL of a site selected from a preset list in response to a statement made by the first avatar.

4. An information processing device according to claim 3, wherein the generation unit extracts user needs from the comments of the first avatar and selects the URL of the site from the list by keyword search based on the user needs.

5. The information processing device according to claim 1, wherein the output unit causes the second avatar to speak a sentence including the URL as the final sentence, and outputs data for ending the conversation.

6. An information processing device according to claim 1, further comprising a conversation control unit that starts the conversation when the first avatar enters within a certain range based on the second avatar.

7. The information processing device according to claim 6, wherein the certain range has a shape in which the front of the second avatar is wider than the rear.

8. The information processing device according to claim 6, wherein the conversation control unit resets the conversation when the first avatar moves out of the certain range.

9. An information processing method comprising the steps of: a computer determining whether a conversation between a first avatar operated by a user in a virtual space and a second avatar acting autonomously contains content that conforms to a set scenario; and, if it is determined that the conversation contains content that conforms to the scenario, outputting data to cause the second avatar to speak a sentence that includes the URL of a set site.

Images