Program, information processing apparatus, and information processing method

Abstract

To improve the relationship between a user and a virtual assistant.SOLUTION: A program causes a computer to realize: a receiving function of receiving, from a user, setting information for setting the operation and / or output mode of a digital assistant that supports handling of a plurality of programs; a first registration function of registering the setting information in a first storage unit; a cooperation function of causing the plurality of programs and the digital assistant to cooperate with each other; an assistant control function of referring to the first storage unit and outputting and actuating the digital assistant in the computer on the basis of the setting information; a second registration function of registering, in a second storage unit, assistant history information indicating the history of output and operation of the digital assistant; and an update function of referring to the second storage unit and updating the setting information on the basis of the assistant history information.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing device, and an information processing method. [Background technology]

[0002] Conventionally, there is known a technology related to a digital assistant that assists the user in operating a program. The virtual assistant device of Patent Document 1 displays an image of a virtual assistant (character) when a user inputs a message, and outputs a message from the virtual assistant. In addition, when the user inputs a word related to the displayed topic image, the virtual assistant device outputs a message related to the word as a message from the virtual assistant. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-176404 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the virtual assistant device of Patent Document 1, the virtual assistant and the user can talk about common topics through topic images, etc., and a relationship can be built between them, but once the conversation ends, the relationship built is lost. For this reason, there is room for improvement in the relationship between the user and the digital assistant in the virtual assistant device of Patent Document 1.

[0005] Therefore, in order to solve the above problems, the present invention aims to improve the relationship between users and digital assistants. [Means for solving the problem]

[0006] A program according to one embodiment of the present invention provides a computer with a reception function that receives setting information from a user to set the movement and / or output manner of a digital assistant that assists in the operation of each of a plurality of programs; a first registration function that registers the setting information in a first memory unit; a linking function that links each of the plurality of programs with the digital assistant; and an assistant control function that refers to the first memory unit and causes the computer to output and operate the digital assistant based on the setting information; a second registration function that registers assistant history information that indicates the history of the output and operation of the digital assistant in a second memory unit; and an update function that refers to the second memory unit and updates the setting information based on the assistant history information.

[0007] An information processing device according to one embodiment of the present invention includes a reception unit that receives setting information from a user for setting the movement and / or output manner of a digital assistant that assists in the operation of each of a plurality of programs; a first registration unit that registers the setting information in a first memory unit; a linking unit that links each of the plurality of programs with the digital assistant; an assistant control unit that references the first memory unit and causes the digital assistant to output and operate on a computer based on the setting information; a second registration unit that registers assistant history information indicating the history of the digital assistant's output and operation in a second memory unit; and an update unit that references the second memory unit and updates the setting information based on the assistant history information.

[0008] An information processing method according to one embodiment of the present invention comprises a computer receiving setting information from a user that sets the movement and / or output manner of a digital assistant that assists in the operation of each of a plurality of programs, registering the setting information in a first memory unit, linking each of the plurality of programs with the digital assistant, referring to the first memory unit to cause the computer to output and operate the digital assistant based on the setting information, registering assistant history information indicating the history of the digital assistant's output and operation in a second memory unit, and referring to the second memory unit to update the setting information based on the assistant history information.

[0009] The present invention aims to improve the relationship between a user and a virtual assistant. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the system configuration of an assist system according to a first embodiment. [Figure 2] FIG. 1 is a diagram for explaining an overview of an example of an assist system according to a first embodiment. [Figure 3] FIG. 1 is a diagram for explaining an overview of an example of an assist system according to a first embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of a user device according to the first embodiment. [Figure 5] FIG. 2 is a diagram illustrating an example of the operation of the assist system according to the first embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of the operation of the assist system according to the first embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of the hardware configuration of a user device and a server device according to the first embodiment. [Figure 8] FIG. 10 is a diagram for explaining an overview of an example of an assist system according to a second embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a functional configuration of a user device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment of the present invention will be described with reference to the accompanying drawings. In the drawings, components with the same reference numerals have the same or similar configurations.

[0012] In this invention, the terms "unit," "means," "device," or "system" do not simply mean physical means, but also include cases where the functions of the "unit," "means," "device," or "system" are realized by software. Furthermore, the functions of one "unit," "means," "device," or "system" may be realized by a combination of two or more physical means, devices, or software modules, and the functions of two or more "units," "means," "device," or "system" may be realized by a single physical means, device, or software module.

[0013] [First embodiment] First, a first embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described.

[0014] The assist system 1 according to this embodiment is a system for assisting a user in operating a plurality of programs using a digital assistant personalized for each user. The assist system 1 can personalize the digital assistant, for example, using customization by the user and / or the history of the digital assistant's output and operation. The digital assistant can assist the user in personalization by outputting images, sounds, etc. from the user device 100 based on setting information customized for each user.

[0015] The setting information is information that sets the movement and / or output mode of the digital assistant for each user and / or each digital assistant. The setting information may, for example, define the digital assistant character (in other words, the digital assistant pattern) as a template. That is, the setting information may be registered and managed for each character that a user wants to use. Furthermore, by creating a template in this way, for example, multiple users can sell or purchase the character, and as a result, the initial settings of similar characters can be shared. Furthermore, multiple users can each create their own character by customizing the initial settings. The same character can also be used by the same user across multiple user devices 100.

[0016] The setting information may further be, for example, information for setting the digital assistant for each target program and / or user device 100.

[0017] In this embodiment, the basic functions of the digital assistant are provided in the user device 100, and an example will be described in which the server device 200 has an online wallet function for managing setting information and / or history information, which will be described later.

[0018] <1. System configuration> An example of the system configuration of an assist system 1 according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the assist system 1 includes a first user device 100a used by a first user, a second user device 100b used by a second user, and a server device 200. The first user device 100a and the second user device 100b are collectively referred to as "user devices 100" when there is no need to distinguish between them. The user device 100 and the server device 200 may be communicably connected to, for example, an external system 500.

[0019] [User device] The user device 100 is a device used by a user. The user device 100 is an information processing device capable of communicating with the server device 200, the external system 500, and the like via the network N1. For example, the user device 100 executes a predetermined program (also referred to as a "user program") to connect to the server device 200 and the external system 500 and send and receive assistant information (setting information and history information) and / or user information, output and operate a digital assistant, cooperate with each of a plurality of programs, and receive instructions from the user by voice or operation.

[0020] The history information may include assistant history information, instruction history information, Web history information, program history information (including operation history information and processing history information), and / or purchase history information, etc., which will be described later.

[0021] The user device 100 may be, for example, a smartphone, a laptop, an XR headset (including a head-mounted display), XR glasses, XR goggles, an XR bodysuit (suit-type device), a smart watch with information and communication functions, wired or wireless earphones, a smart speaker with information and communication functions, a smart home appliance with information and communication functions, a robot, etc.

[0022] The user device 100 may be, for example, a vehicle equipped with an information and communication function, such as a so-called smart car or a connected car, or a device such as an electrically assisted bicycle, an electric kickboard, or an electric wheelchair.

[0023] The user program may be an application (such as a so-called native application) dedicated to the assist system 1. As another example, the user program may be, for example, an application program of a third-party system (also referred to as a "third-party application") installed in the user device 100. The third-party application may be, for example, a web browser that is provided as standard in the user device 100, which is a general-purpose terminal device.

[0024] User information is information about a user. User information may include, for example, user identification information for identifying a user, and so-called personal information of the user, such as the user's name, telephone number, address, email address, personal identification number (My Number), gender, and / or date of birth.

[0025] The user information may include, for example, the user's license number, card number, account information of the account held by the user (for example, account type, account number, name, etc.), and the like.

[0026] The user information may include, for example, biometric information relating to the biometric characteristics of the user (for example, facial image data, fingerprint information, iris information, or voiceprint information of the user).

[0027] The user information may include, for example, location information (e.g., GPS data, etc.), web history information (e.g., browsing history, search history, cookie information, etc.), operation history information indicating the operation history of devices including the user device 100 or processing history information indicating the processing history, and / or purchase history information of products or services on an e-commerce site.

[0028] [Server device] The server device 200 is an information processing device capable of communicating with the user device 100 and / or the external system 500 via the network N1. The server device 200 executes a predetermined program (also referred to as a "server program") to transmit and receive user information, setting information, and / or history information from the user device 100, and to register and manage this information in a memory unit of the server device 200 or an external device.

[0029] [network] The network N is configured by a wireless network or a wired network. Examples of the network N include a mobile phone network, a PHS (Personal Handy-phone System) network, a wireless LAN (a local area network, including communication conforming to IEEE802.11 (so-called Wi-Fi (registered trademark))), 3G (3rd Generation), LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation), WiMax (registered trademark), infrared communication, visible light communication, Bluetooth (registered trademark), a wired LAN, a telephone line, a power line communication, a power line network, a network conforming to IEEE1394, etc.

[0030] [External System] The external system 500 is a so-called third-party system. The external system 500 is communicably connected to the user device 100 and / or the server device 200 via, for example, a network N. The external system 500 may include, for example, two types of systems: (1) a system that performs some of the functions of the digital assistant, and (2) a system of a program that is the target of support by the digital assistant.

[0031] In the case of (1) above, the external system 500 may include, for example, a system that provides an ASP / SaaS service using a language model (specifically, a dialogue AI, a language generation AI, or the like). Specifically, the server device 200 transmits instructions for the language model to the external system 500 via API integration or the like, and receives processing results for these instructions from the external system 500. Furthermore, in the case of (2) above, the external system 500 may be any system that can be integrated with a digital assistant, and an external interface between the program of the external system 500 and a user program or a server program, which is required for integration, may be defined in advance.

[0032] The external system 500 may include, for example, an online storage system that stores setting information and / or history information used by the assistant system 1.

[0033] The external system 500 may include, for example, a system for distributing setting information, in other words, a system for conducting BtoC (Business to Consumer) transactions of setting information between companies and consumers, or CtoC (Consumer to Consumer) transactions of setting information between consumer users (also called a "transaction system").

[0034] <2. Overview> An example of the outline of the assist system 1 will be described with reference to FIGS. 2 and 3. FIG. 2 describes an example of a situation in which pre-settings for using a digital assistant are performed. FIG. 3 describes an example of a situation in which a user uses a digital assistant. In the example of FIGS. 2 and 3, the programs to be assisted (also referred to as "target programs") are a game program A1 in a game system 500a and a spreadsheet program B1 in a spreadsheet system (spreadsheet software) 500b. The game program A1 and the spreadsheet program B1 may be arranged in any manner as long as they can be linked to the user program (personalized assistant function). For example, they may be provided (downloaded) in the user device 100, or some or all of them may be provided in each server connected via the network N. In addition, in this example, an example will be described in which the digital assistant is an avatar.

[0035] As shown in FIGS. 2 and 3, the personalized assistant functions realized by the assistant system 1 are provided in the user device 100 and may be classified as follows: In addition, some functions such as setting and history functions and / or linking functions may also be provided in the server device 200, or instead of them. In the assistant system 1, for example, the classified functions are linked together (including information linking and / or function linking; the same applies below). * Settings and history function (wallet): Stores information such as user information, settings information and / or history information for each user. User settings management function: Manages the information stored in the settings and history memory function. Assistant generation and control function: Generate and control digital assistants, including avatars. 〇 Collaboration function: Links the target program with the digital assistant. UI function: Provides output to the user of the assistant system 1, including the avatar.

[0036] <2-1. Pre-settings>

[0037] (1) As shown in Figure 2, a user requests registration of a new account and / or setting up a digital assistant. This registration request includes user information to be registered, and this setting request includes setting information for an avatar that will assist in the operation of each of the game program A1 and spreadsheet program B1 to be registered. User device 100 accepts the user information and setting information.

[0038] The setting information may include, for example, the following information regarding the output of the digital assistant: Appearance information: When displaying images, videos, etc. of a digital assistant, images, videos, and / or operation information that realize the basic appearance, shape, and behavior, etc., or data that can be incorporated into them. - Voice information: When providing a digital assistant to a user via voice, this includes voice quality, basic voice information, or data that can be incorporated into these.

[0039] The setting information may include, for example, the following information regarding the operation of the digital assistant: - Expression information: When a digital assistant is presented to a user in text and / or voice, etc., language settings, wording settings, response habits, or data that can be incorporated into these. Relationship information: Information that defines the relationship between the user and the digital assistant when the user and the digital assistant interact. Specifically, the relationship information may define that the digital assistant is a pet or fairy that accompanies the user, a team member (party) who plays a game such as a role-playing game together, the user's virtual boyfriend or girlfriend, etc. The relationship information may also include an intimacy level that indicates the depth of the relationship between the user and the digital assistant. Scenario information: Background information that indicates the background in which the user is placed when the digital assistant communicates with the user, scenario information that controls the development of the scene and / or the behavior of the digital assistant according to the scene, and / or virtual space data that generates the virtual space in which the digital assistant exists. Insertion information: Information to realize games, movies, and / or events that are inserted into the user's application operation.

[0040] The setting information may include, for example, program information about the target program (e.g., program name, program type, version, etc.), other information about the digital assistant (e.g., digital assistant type, version information, etc.), device information about the user device 100 that uses the digital assistant, etc.

[0041] The device information is information about each device including the user device 100. The device information may include, for example, device identification information for identifying each device, the device type, product name or product number, MAC address, IP address, and / or serial number. The device information may also include, for example, program information about one or more application programs installed on each device.

[0042] (2) When the user selects, as the setting and history function, a local wallet, i.e., edge-side management of settings and history, the user device 100 registers the user information and setting information received in (1) above in the first user storage unit 131. On the other hand, when the user selects, as the setting and history function, an online wallet, i.e., cloud-side management of settings and history, the user device 100 transmits to the server device 200 a registration request including the user information received in (1) above and a setting request including the setting information.

[0043] (3) When the server device 200 receives the above registration request and setting request from the user device 100 as a setting and history function, it registers the request in the first server storage unit 231 .

[0044] (4) As a linking function, the user device 100 requests the game program A1 of the game system 500a and the spreadsheet program B1 of the spreadsheet system B to perform pre-settings for linking with the digital assistant. In response to this request and as pre-settings for the link between the game program A1 and the user program, the game program A1 may set the user program as a recipient of information indicating the program's operating status (also referred to as "operating status information"), such as output parameters and logs. In response to this request and as pre-settings for the link between the spreadsheet program B1 and the user program, the user program may be added as a function expansion program using a so-called add-on, add-in, or plug-in. In this case, the user device 100 may perform settings for adding or enabling the user program as an add-on, add-in, or plug-in to the spreadsheet program B1.

[0045] (5) The user device 100 may delegate all or part of the avatar-related functions to the AI ​​assistant system 500c as an assistant generation and control function. In this case, the user device 100 may specify a role (e.g., to be the user's companion pet or a virtual boyfriend / girlfriend) as a pre-setting request based on relationship information and / or scenario information included in the setting information.

[0046] <2-2. Support using avatars> (1) The user requests the user device 100 to start (launch) the game program A1 or the spreadsheet program B1 by operating the device. This request may also serve as a call to an avatar, or the avatar may be called by a separate operating input. The user device 100 accepts the request as a UI function.

[0047] (2) As a setting and history function, the user device 100 reads setting information, history information, etc. managed in a local wallet. Also, when setting information, history information, etc. are managed in an online wallet, the user device 100, as a setting and history function, requests the server device 200 to provide this information, and acquires this information provided from the server device 200 as a response to the provision request.

[0048] As an assistant generation and control function, the user device 100 instructs the AI ​​assistant system 500c to generate lines (sentences) for the avatar to speak to the user based on the loaded or acquired setting information, history information, etc. (sends prompts). The AI ​​assistant system 500c generates the lines based on this generation instruction. The user device 100 acquires the generated lines (text information) from the AI ​​assistant system 500c in response to the generation instruction. Note that this series of processes of instructing to generate lines, generating lines, and acquiring the generated lines may be repeated depending on the operating conditions, such as input from the user and / or parameters acquired from the target program, until collaboration ends, the avatar's operation ends, or the target program's operation ends.

[0049] (3) As a linking function, the user device 100 links the target program start request received from the user to the game program A1 and the spreadsheet program B1. If the game program A1 and the spreadsheet program B1 are installed on an external device, the user device 100 may establish a communication connection to link them. The game program A1 and the spreadsheet program B1 each start operating based on the linked start request. Thereafter, until the linking ends, the operation of the avatar ends, or the operation of the target program ends, the user device 100 links the game program A1 and the spreadsheet program B1 with the avatar (specifically, the avatar generation and control function).

[0050] (4) As an assistant generation and control function, the user device 100 generates an avatar based on the acquired information indicating the lines, setting information, material information indicating materials for generating and controlling the avatar, and / or history information. Information about the digital assistant acquired from an external device (information indicating the lines), setting information, and material information are collectively referred to as "assistant information." Assistant information is information about the digital assistant.

[0051] User device 100 outputs and operates the generated avatar based on the acquired assistant information and / or history information, etc., in cooperation with the target program (specifically, in accordance with information such as the operating status of the cooperated target program). For example, when cooperation is established that an operation related to adding a graph is being performed from spreadsheet program B1, user device 100 may display and output audibly a URL to a help screen related to adding a graph and a line from the avatar, including a line such as "If you're having trouble adding a graph, try clicking this URL!", together with the screen display of spreadsheet program B1.

[0052] The user device 100, as a UI function, receives instructions (including, for example, questions and simple prompts) from the user for the output avatar. The user device 100, as an assistant generation / control function, may output and operate the avatar based on the instructions.

[0053] The material information may be, for example, at least one of text information (including HTML data, etc.), still or video image data, audio data, and / or executable files, etc. For example, if the digital assistant is an avatar, the material information may include an image of the avatar's character, information about the polygon modeling and texture of the body and clothing, information about the animation and motion that realizes the avatar's movements, and / or audio data that outputs the avatar's voice.

[0054] The material information may be, for example, information that stimulates the user's five senses (sight, hearing, taste, smell, and touch). In the case of the user's sense of touch, the material information may be information for instructing the VR body suit, which is the user device 100, to move or information for causing the user device 100 to vibrate.

[0055] (5) As an assistant generation / control function and a setting / history function, the user device 100 registers assistant history information indicating the output and operation history of the digital assistant executed in (4) above in the second user storage unit 132 or the second server storage unit 232 of the server device 200. In addition, the user device 100 registers instruction history information indicating the history of the user's instructions to the avatar in the third user storage unit 133 or the third server storage unit 233 of the server device 200.

[0056] (6) The user device 100 or the server device 200 updates the setting information based on the registered history information. The next time the avatar is called, the user device 100 outputs and operates the avatar based on the updated setting information, thereby supporting the user's operation while maintaining and further developing the relationship that has been built up to a certain extent, including the previous time.

[0057] With the above configuration, the assist system 1 manages history information related to the digital assistant and updates the setting information using this history information, enabling a natural story development the next time the digital assistant is used. This improves the relationship between the user and the digital assistant, making the digital assistant more like a friend or buddy.

[0058] With the above configuration, the assist system 1 can use a common digital assistant for multiple different programs (game program A1 and spreadsheet program B1), allowing the user and the digital assistant to interact across both programs. This allows for more comprehensive and multifaceted support for the user, thereby improving the relationship between the user and the digital assistant.

[0059] <3. Functional configuration> The functional configuration of the user device 100 according to this embodiment will be described with reference to Fig. 4. As shown in Fig. 4, the user device 100 includes a user control unit 110, a user communication unit 120, and a user storage unit 130.

[0060] The user control unit 110 may include a user reception unit 111 (one embodiment of the reception unit), a user registration unit 112 (one embodiment of the registration unit), a user collaboration unit 113 (one embodiment of the collaboration unit), a user assistant control unit 114 (one embodiment of the assistant control unit), a user update unit 115 (one embodiment of the update unit), a user instruction unit 116 (one embodiment of the instruction unit), and / or a user output unit 117 (one embodiment of the output unit).

[0061] The user acceptance unit 111, as a user setting management function, a linkage function, and / or a UI function, accepts various information and instructions from the server device 200 and / or devices in the external system 500 via the user communication unit 120 in a cyclic or event-driven manner. The user acceptance unit 111 may accept various information and instructions in any manner. For example, the user acceptance unit 111 may receive a file indicating operation status information of a program installed on the device in the external system 500 from the device. For example, when a user inputs user information into a web browser screen from the user device 100, the user acceptance unit 111 may receive a message indicating the input user information as needed. For example, the user acceptance unit 111 may instruct an API implemented in the device from which the information is obtained to reference various information and accept various information as a result of the reference. The user acceptance unit 111 may also accept various information using an SDK library including an API.

[0062] The user reception unit 111 receives, from the user, setting information for setting the movement and / or output mode of the digital assistant that supports the operation of each of the multiple programs. The user reception unit 111 may also receive, for example, from the user, an update to the setting information registered in the first user storage unit 131.

[0063] The user reception unit 111 may, for example, receive instructions for the digital assistant from the user. The user reception unit 111 may also, for example, receive operation inputs for each of the plurality of target programs from the user in order to mediate between the plurality of target programs.

[0064] The user receiving unit 111 may, for example, receive from the user a designation of a group for two or more programs among a plurality of target programs, and may receive setting information for each designated group. The unit of grouping may be, for example, (a) for each digital assistant, (b) for each type of program (e.g., game, SNS, schedule management, chat / mail software, or text editor), or (c) for each user device 100.

[0065] For example, when grouping is performed by the digital assistant in (A) above and the target programs include program D, program E, program F, and program G, the user reception unit 111 may first receive a designation of "pet XX (group a)" and "virtual girlfriend XX (group b)." Then, the user reception unit 111 may receive a designation that program D and program E are group a, and program F and program G are group b.

[0066] The user reception unit 111 may receive, for example, program history information indicating the operation history of each of the plurality of target programs from each of the plurality of target programs.

[0067] The user receiving unit 111 may, for example, receive a request from a user to provide the setting information of the user's digital assistant to a user other than the user (also referred to as a "recipient user"). For example, when providing (selling) the setting information through a trading system, the request may be a request to put the information up for sale in the trading system.

[0068] The user reception unit 111 may include, for example, a user acquisition unit 111a. The user acquisition unit 111a acquires provision result information indicating the result of the provision of the setting information from a providing device for providing the setting information to the destination user, as a response to an instruction to provide the setting information. The providing device may be, for example, a device of a trading system or its own device (user device 100) when a transaction (provision) is conducted between a user and a destination user via a trading system. Furthermore, the providing device may be the server device 200 when the user provides the setting information to the destination user via the server device 200. Furthermore, when the user provides the setting information directly to the destination user, the providing device may be its own device.

[0069] The user registration unit 112 registers various pieces of information received by the user reception unit 111 in the user storage unit 130 or the like as a setting / history function or the like.

[0070] The user registration unit 112 may include, for example, a first user registration unit 112a (one aspect of the first registration unit). The first user registration unit 112a registers the setting information accepted by the user acceptance unit 11 in the first user storage unit 131 for each user.

[0071] The first user registration unit 112a may register the setting information for each group, for example.

[0072] The user registration unit 112 may include, for example, a second user registration unit 112b (one aspect of the second registration unit). The second user registration unit 112b registers assistant history information indicating the history of the output and operation of the digital assistant in the second user storage unit 132. The second user registration unit 112b may also register, for example, instruction history information indicating the history of instructions from the user in the third user storage unit 133.

[0073] The user registration unit 112 may include, for example, a user adjustment unit 112c (one aspect of the adjustment unit). The user adjustment unit 112c may update the setting information based on a combination of at least two of the target program, the digital assistant, and the user device 100.

[0074] The user linking unit 113 links each of the multiple target programs with the digital assistant via the user communication unit 120.

[0075] The user assistant control unit 114 references the first user storage unit 131 and generates, outputs, and operates a digital assistant in the user device 100 based on the setting information. First, when generating and outputting a digital assistant character, the user assistant control unit 114 may read appearance information, audio information, relationship information, and material information to generate and model an image of the character. Then, the user assistant control unit 114 may output lines or make gestures or other movements of the modeled character based on expression information, scenario information, and the like.

[0076] For example, the user assistant control unit 114 may appropriately use the AI ​​assistant system 500c for outputting and operating the digital assistant. Also, the user assistant control unit 114 may generate and output the digital assistant using a learned language model (e.g., a conversational AI) and / or an image generation model stored in the user memory unit 130.

[0077] For example, when displaying and outputting the digital assistant, the user assistant control unit 114 may output it in the same display area, such as the same window as the target program, or may output it in a different display area, such as a different window or virtual desktop, and display it superimposed or alongside the program display.

[0078] The user assistant control unit 114 may, for example, output and operate the digital assistant further based on instructions from the user.

[0079] The user update unit 115 may update the setting information based on a request for updating the setting information received by the user reception unit 111 .

[0080] The user update unit 115 may, for example, refer to the second user storage unit 132 and update the setting information based on the assistant history information.

[0081] For example, if the line output by the digital assistant when completing the operation of the target study program is "Thank you for your hard work. You studied for three hours today (4 / 24). I'll keep encouraging you to study even harder next time!", the user update unit 115 may update the scenario information based on the history of this line to record a story development such as "Output encouragement for studying for three hours on 4 / 24 and for the next study: When the user next starts up, praise him for studying for three hours on 4 / 24 and encourage him to study for another hour." Based on this updated setting information, the user assistant control unit 114 may display and / or output as audio the line of the digital assistant when the study preparation program and the digital assistant are next started up.

[0082] According to the above configuration, the next digital assistant can output and operate in a manner that inherits the previous interaction that the digital assistant had with the user through a history update. By updating in this way, it is possible to personalize the digital assistant for each user, and by not interrupting the interaction, it is possible to develop a story and further deepen the relationship between the user and the digital assistant.

[0083] The user update unit 115 may, for example, refer to the third user storage unit 133 and update the setting information based on the instruction history information. For example, if there is a history of a user inputting an instruction to the digital assistant, such as "Teach me how to operate the word processing software!" while the digital assistant is assisting the user in operating a word processing software program, the user update unit 115 may update the relationship information to record "user's word processing software teacher." Based on this updated relationship information and appearance information, the user assistant control unit 114 may change the image of the digital assistant character to look like a teacher by wearing glasses, a suit, or a wand.

[0084] According to the above configuration, the user's previous instructions can be fed back to the digital assistant from the next time onwards. This allows the digital assistant to be personalized for each user, and the story can be continued. This deepens the relationship between the user and the digital assistant.

[0085] For example, the user update unit 115 may refer to the fourth user storage unit 134, which stores program history information indicating the operation history of each of multiple target programs, and update the setting information based on this program history information. For example, if the program history information for a target game program indicates that a specific part A of the game was cleared while wearing specific equipment and the player's level increased by 10, the user update unit 115 may update the appearance information so that the digital assistant's costume resembles this equipment (for example, if it is a hero's equipment, the digital assistant will look like the hero). The user assistant control unit 114 may output the digital assistant in this updated costume when operating the game program and digital assistant.

[0086] According to the above configuration, the history information output from the target program can be used to provide feedback on the performance of the target program to the digital assistant. This allows a connection between the target program and the digital assistant to be created. This allows the relationship between the user and the digital assistant to be deepened through the target program.

[0087] For example, when setting information is registered for each group, the user update unit 115 may update the setting information for each group based on the program history information by referring to the third user storage unit 133. For example, the user update unit 115 may update the setting information of a specific group with the program history information of the program J, program K, and game program L grouped in this group, while not updating with the program history information of programs in different groups.

[0088] According to the above configuration, it is possible to manage and continuously update the setting information for each group specified for the digital assistant.

[0089] The user instruction unit 116 issues various instructions to the server device 200 and / or devices in the external system 500. For example, based on a request to provide setting information, the user instruction unit 116 may issue a provision instruction to the providing device to acquire the user's setting information from the first user storage unit 131 or the first server storage unit 231 and provide the setting information to the destination user.

[0090] According to the above configuration, the setting information of one's digital assistant can be provided to other users, such as friends. This allows the digital assistant to be shared with other users and to benefit from the provision, increasing the interest in using the digital assistant. In addition, the other users who have received the setting information can personalize the digital assistant by updating the setting information themselves.

[0091] The user output unit 117 outputs various information related to the assist system 1 by screen display, audio output, and / or file output, etc. The user output unit 117 outputs the results of provision to the user, for example, based on the provision result information acquired by the user acquisition unit 111a.

[0092] For example, the user control unit 110 may provide the setting information registered in the first user storage unit 131 to the user device 100 of the destination user based on an instruction to provide the setting information by the user instruction unit 116. Furthermore, if the provision is a transaction, the user control unit 110 may execute processing related to the transaction (for example, a settlement process for paying for the setting information, etc.). The user control unit 110 may generate provision result information indicating the result of this provision.

[0093] The user communication unit 120 transmits and receives various information and / or instructions to and from the server device 200 and / or devices of the external system 500 via the network N.

[0094] The user storage unit 130 may store user information in association with setting information and history information as a local wallet, for example. The user storage unit 130 may include, for example, a first user storage unit 131, a second user storage unit 132, a third user storage unit 133, and / or a fourth user storage unit 134.

[0095] The first user storage unit 131 stores setting information for each user (further, for each digital assistant and / or for each group). The second user storage unit 132 stores assistant history information for each user (further, for each digital assistant and / or for each group). The third user storage unit 133 stores instruction history information for each user (further, for each digital assistant and / or for each group). The fourth user storage unit 134 stores program history information for each user (further, for each digital assistant and / or for each group). The setting information and this history information may be stored in association with each other.

[0096] <4. Example of operation> An example of the operation of the assist system 1 will be described with reference to FIGS. 5 and 6. FIG. 5 is a flow diagram showing the processing flow of the user device 100 for presetting to use the digital assistant. FIG. 6 is a flow diagram showing the processing flow of the user device 100 when using the digital assistant. Note that the order and grouping of the processes shown below are examples and may be changed as appropriate. Also, the processing flow shown below does not have to be a series, and some of the processes may be processed in parallel.

[0097] 5, user reception unit 111 receives setting information for a digital assistant that supports the operation of each of a plurality of programs from a user (S10). First user registration unit 112a registers this setting information in first user storage unit 131 (S11). User association unit 113 associates each of the plurality of programs with the digital assistant (S12).

[0098] 6, the user reception unit 111 receives a start instruction for the digital assistant from the user (S20). The user assistant control unit 114 references the first user storage unit 131 and, based on the start instruction and the assistant information including the setting information, causes the digital assistant to output and start operating in the user device 100 so as to work in conjunction with each of the multiple programs (S21).

[0099] The user assistant control unit 114 refers to the first user storage unit 131 and causes the digital assistant to output and operate in conjunction with each of the multiple programs on the user device 100 based on the assistant information (S22). In parallel with the output and operation of the digital assistant, the second user registration unit 112b registers assistant history information indicating the history of the output and operation of the digital assistant in the second user storage unit 132 (S23).

[0100] The user reception unit 111 receives an instruction from the user to terminate the digital assistant (S24). The user assistant control unit 114 terminates the output and operation of the digital assistant so as to link with each of the multiple programs in the user device 100 in response to user operation inputs, etc. (S25). The user update unit 115 refers to the second user storage unit 132 and updates the setting information based on the assistant history information (S26). Returning to the processing of step S20, if an instruction to start the digital assistant is received from the user again, steps S21 to S26 are repeated.

[0101] <5. Hardware Configuration> 7, an example of a hardware configuration in which the above-described user device 100 and server device 200 are realized by a computer 800 will be described. Note that the functions of each device can also be realized by dividing them among multiple devices.

[0102] As shown in FIG. 7, the computer 800 includes a processor 801, a memory 803, a storage device 805, an input I / F unit 807, a data I / F unit 809, a communication I / F unit 811, and a display device 813.

[0103] The processor 801 controls various processes in the computer 800 by executing programs (e.g., user programs and / or server programs) stored in the memory 803. For example, the various functional units of the user control unit 110 of the user device 100 or the server control unit 210 of the server device 200 (described later) can be realized by the processor 801 executing the programs temporarily stored in the memory 803.

[0104] The memory 803 is a storage medium such as a RAM (Random Access Memory), etc. The memory 803 temporarily stores the program code of the program executed by the processor 801 and data required when the program is executed.

[0105] The storage device 805 is a non-volatile storage medium such as a hard disk drive (HDD) or flash memory. The storage device 805 stores an operating system and various programs for implementing the above-mentioned configurations. In addition, the storage device 805 can also store tables for registering user information, setting information, and / or history information, and a DB for managing the tables. Such programs or data are loaded into the memory 803 as needed and can be referenced by the processor 801.

[0106] The input I / F unit 807 is a device for receiving input from a user. Specific examples of the input I / F unit 807 include a keyboard, a mouse, a touch panel, various sensors, and a wearable device. The input I / F unit 807 may be connected to the computer 800 via an interface such as a USB (Universal Serial Bus).

[0107] The data I / F unit 809 is a device for inputting data from outside the computer 800. A specific example of the data I / F unit 809 is a drive device for reading data stored in various storage media. The data I / F unit 809 may be provided outside the computer 800. In this case, the data I / F unit 809 is connected to the computer 800 via an interface such as a USB.

[0108] The communication I / F unit 811 is a device for performing data communication via a network N, either wired or wirelessly, with an external device of the computer 800. The communication I / F unit 811 may be provided outside the computer 800. In this case, the communication I / F unit 811 is connected to the computer 800 via an interface such as a USB.

[0109] The display device 813 is a device for displaying various types of information. Specific examples of the display device 813 include a liquid crystal display, an organic EL (Electro-Luminescence) display, and a display of a wearable device. The display device 813 may be provided outside the computer 800. In this case, the display device 813 is connected to the computer 800 via, for example, a display cable. Furthermore, when a touch panel is adopted as the input I / F unit 807, the display device 813 can be configured as an integral part of the input I / F unit 807.

[0110] [Second embodiment] Next, a second embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described.

[0111] In the first embodiment, an example was described in which the basic functions of the digital assistant are provided in the user device 100 and the server device 200 is provided with an online wallet function, but in this embodiment, an example will be described in which the basic functions of the digital assistant are also provided in the server device 200. The following will mainly describe the differences from the first embodiment.

[0112] <1. Overview of avatar support on server devices> As shown in Fig. 2, the server device 200 has personalized assistant functions (settings and history function (wallet), user settings management function, assistant generation and control function, collaboration function, UI function). Note that the user device 100 may also have some of the functions, such as the settings and history function (wallet) and / or the UI function, in addition to or instead of the functions.

[0113] (1) The user device 100 requests the user device 100 to start (launch) the game program A1 or the spreadsheet program B1 through an operation input. The server device 200 receives the request as a UI function.

[0114] (2) When the user selects to manage the settings and history using an online wallet, i.e., on the cloud side, the server device 200 reads the setting information, history information, etc. registered in the online wallet, the first server storage unit 231. When the user selects to manage the settings and history using a local wallet, i.e., on the edge side, and the setting information, history information, etc. are managed in the first user storage unit 131 of the user device 100, which is a local wallet, the server device 200, as the setting and history function, requests the user device 100 to provide this information, and acquires this information provided by the user device 100 in response to the provision request.

[0115] As an assistant generation and control function, the server device 200 instructs the AI ​​assistant system 500c to generate lines (text) for the avatar to speak to the user based on the loaded or acquired setting information, history information, etc. The AI ​​assistant system 500c generates the lines based on the instructions. The server device 200 obtains the generated lines (text information) from the AI ​​assistant system 500c in response to the instructions.

[0116] (3) As a linking function, the server device 200 links the target program start request received from the user device 100 to the game program A1 and the spreadsheet program B1. The game program A1 and the spreadsheet program B1 each start operating based on the linked start request. Thereafter, the server device 200 links the avatar with the game program A1 and the spreadsheet program B1 until the linking ends, the avatar's operation ends, or the target program operation ends.

[0117] (4) As an assistant generation / control function, the server device 200 generates an avatar based on assistant information including information indicating the acquired lines, setting information, material information indicating materials for generating / controlling the avatar, and / or history information.

[0118] The server device 200 outputs and operates the generated avatar based on the acquired assistant information and / or history information, etc., in cooperation with the target program.

[0119] The server device 200, as a UI function, receives instructions from the user for the output avatar. The server device 200, as an assistant generation / control function, may output and operate the avatar based on the instructions.

[0120] (5) As an assistant generation / control function and a setting / history function, the server device 200 registers assistant history information indicating the output and operation history of the digital assistant executed in (4) above in the second server storage unit 232 (online wallet) or the second user storage unit 132 (local wallet) of the user device 100. In addition, the server device 200 registers instruction history information indicating the history of the user's instructions to the avatar in the third server storage unit 233 (online wallet) or the third user storage unit 133 (local wallet) of the user device 100.

[0121] (6) The server device 200 updates the setting information based on the registered history information.

[0122] <2. Functional configuration of the server device> The functional configuration of the server device 200 according to this embodiment will be described with reference to Fig. 9. As shown in Fig. 9, the server device 200 includes a server control unit 210, a server communication unit 220, and a server storage unit 230. These units have functions similar to those of the user control unit 110, the user communication unit 120, and the user storage unit 130 of the user device 100 according to the first embodiment, for example.

[0123] The server control unit 210 may include a server reception unit 211 (one embodiment of the reception unit), a server registration unit 212 (one embodiment of the registration unit), a server collaboration unit 213 (one embodiment of the collaboration unit), a server assistant control unit 214 (one embodiment of the assistant control unit), a server update unit 215 (one embodiment of the update unit), a server instruction unit 216 (one embodiment of the instruction unit), and / or a server output unit 217 (one embodiment of the output unit). These units may have functions similar to those of the user reception unit 111, the user registration unit 112, the user collaboration unit 113, the user assistant control unit 114, the user update unit 115, the user instruction unit 116, and the user output unit 11 of the user control unit 110 of the first embodiment, for example.

[0124] The server reception unit 211 receives, for example, from each of the user devices 100 of multiple users, setting information for setting the movement and / or output mode of a digital assistant that supports the operation of each of multiple programs included in the user device 100. The server reception unit 211 may also include, for example, a server acquisition unit 211a. The server acquisition unit 211a has the same function as the user acquisition unit 111a of the user device 100.

[0125] The server registration unit 212 may include a first server registration unit 212a, a second server registration unit 212b, and / or a server adjustment unit 212c. These units have functions similar to those of the first user registration unit 112a, the second user registration unit 112b, and the user adjustment unit 112c of the user device 100 of the first embodiment, for example. The first server registration unit 212a registers setting information for each user in the first server storage unit 231.

[0126] The server linking unit 213 links each of the multiple programs with the digital assistant. The server assistant control unit 214 references the first storage unit and outputs and operates the digital assistant on the user device 100 based on the link between each of the multiple programs and the avatar and the setting information registered in the first server storage unit 231.

[0127] Second server registration unit 212b registers assistant history information indicating the output and operation history of the digital assistant for each user in second server storage unit 232. Server update unit 215 updates the setting information registered in second server storage unit 232 for each user based on the assistant history information.

[0128] The above-described embodiments are merely examples for explaining the present invention, and are not intended to limit the present invention to these embodiments. Furthermore, the present invention can be modified in various ways without departing from the spirit of the invention. Furthermore, those skilled in the art can adopt embodiments in which the elements described below are replaced with equivalents, and such embodiments are also within the scope of the present invention.

[0129] The components of the server device 200 described in the above embodiment are assumed to realize predetermined processing in cooperation with other hardware by the processor 801 executing a program stored in the storage device 805. In other words, these components are considered to be software or firmware, as well as corresponding hardware, and in both of these concepts, they are also referred to as "functions," "means," "parts," "processing circuits," "units," or "modules," and can be interpreted as such. [Explanation of symbols]

[0130] 1...assist system, 100...user device, 110...user control unit, 120...user communication unit, 130...user memory unit, 200...server device, 210...server control unit, 220...server communication unit, 230...server memory unit, 500...external system, 800...computer, 801...processor, 803...memory, 805...storage device, 807...input I / F unit, 809...data I / F unit, 811...communication I / F unit, 813...display device.

Claims

1. On the computer, A reception function that receives setting information from a user to set the movement and / or output mode of a digital assistant that assists in the operation of each of a plurality of programs; a first registration function for registering the setting information in a first storage unit; A linking function that links each of the plurality of programs with the digital assistant; An assistant control function that refers to the first storage unit and outputs and operates the digital assistant in the computer based on the setting information; A second registration function that registers assistant history information indicating the output and operation history of the digital assistant in a second storage unit; and an update function of updating the setting information based on the assistant history information by referring to the second storage unit. program.

2. The reception function receives instructions from the user to the digital assistant, The assistant control function outputs and operates the digital assistant further based on the instruction, the second registration function registers instruction history information indicating a history of the instruction in a third storage unit; the update function updates the setting information further based on the instruction history information. The program according to claim 1.

3. the update function refers to a fourth storage unit that stores program history information indicating an operation history of each of the plurality of programs, and updates the setting information further based on the program history information. The program according to claim 1.

4. the receiving function receives, from the user, designation of groups for two or more programs among the plurality of programs, and receives the setting information for each of the designated groups; the first registration function registers the setting information for each of the groups; the update function refers to the fourth storage unit and updates the setting information for each group further based on the program history information. The program according to claim 3.

5. The reception function receives a provision request from the user to provide the setting information of the user's digital assistant to a destination user different from the user, The computer, an instruction function for instructing a providing device to acquire the setting information of the user from the first storage unit and provide the setting information to the destination user based on the provision request; an acquisition function that acquires, from the providing device, provision result information indicating a result of the provision as a response to the provision instruction; and an output function of outputting the result of the provision to the user based on the provision result information. The program according to claim 1 or 2.

6. On the computer, A reception function that receives setting information from each user device of a plurality of users to set the movement and / or output mode of a digital assistant that supports the operation of each of a plurality of programs provided in the user device; a first registration function for registering the setting information in a first storage unit for each user; A linking function that links each of the plurality of programs with the digital assistant; By referring to the first storage unit, an assistant control function is realized in the user device that outputs and operates the digital assistant based on the cooperation between each of the plurality of programs and the digital assistant and the registered setting information, A second registration function that registers assistant history information indicating the output and operation history of the digital assistant for each user in a second storage unit; and an update function for updating the setting information for each user based on the assistant history information. program.

7. A reception unit that receives setting information from a user that sets the movement and / or output mode of a digital assistant that assists in the operation of each of a plurality of programs; a first registration unit that registers the setting information in a first storage unit; A linking unit that links each of the plurality of programs with the digital assistant; An assistant control unit that refers to the first storage unit and outputs and operates the digital assistant in the information processing device based on the setting information; A second registration unit that registers assistant history information indicating the output and operation history of the digital assistant in a second storage unit; and an update unit that refers to the second storage unit and updates the setting information based on the assistant history information. Information processing device.

8. The computer Accepting setting information from a user that sets the movement and / or output mode of a digital assistant that assists in the operation of each of a plurality of programs; registering the setting information in a first storage unit; Linking each of the plurality of programs with the digital assistant; By referring to the first storage unit, the digital assistant is output and operated in the computer based on the setting information, Register assistant history information indicating the output and operation history of the digital assistant in a second storage unit; updating the setting information based on the assistant history information by referring to the second storage unit; Information processing methods.

Images