Information processing system and information processing method

Abstract

[Problem] To make it possible for conversation messages to be efficiently and effectively generated. [Solution] An information processing system that is characterized by comprising: a message storage unit that stores past messages in association with a user, the past messages being messages transmitted to and received from the user; and a message generation unit that generates a transmission message by assigning, to a large-scale language model, a prompt which includes past messages corresponding to a user who is a conversation partner and an instruction for generating a transmission message which is a message to be newly transmitted to the user, wherein the message generation unit deletes, from the prompt, a portion of the past messages corresponding to the user who is a conversation partner.

Description

Information processing system and information processing method

[0001] The present invention relates to an information processing system and an information processing method.

[0002] A conversation program using artificial intelligence has been provided (see, for example, Patent Document 1).

[0003] Special Publication No. 2022-531994

[0004] There is a need to ensure efficient processing while maintaining conversational consistency.

[0005] The present invention has been made in view of the above background, and has an object to provide a technique that can generate conversational messages efficiently and effectively.

[0006] The main invention of the present invention for solving the above problem is an information processing system comprising: a message storage unit that stores past messages, which are messages sent and received between a user and a user, in association with the user; and a message generation unit that generates the outgoing message by providing a large-scale language model with a prompt that includes the past message corresponding to the user who is a dialogue partner and an instruction to generate an outgoing message, which is a message to be newly sent to the user, and is characterized in that the message generation unit deletes a portion of the past message corresponding to the user who is a dialogue partner from the prompt.

[0007] Other problems and solutions disclosed in this application will be made clear in the section on preferred embodiments of the invention and the drawings.

[0008] According to the present invention, conversational messages can be generated efficiently and effectively.

[0009] It is a diagram showing an example of the overall configuration of an information processing system. It is a diagram showing an example of the hardware configuration of a management server 2. It is a diagram showing an example of the software configuration of a management server 2. It is a diagram explaining the operation of a management server 2.

[0010] <System Overview> An information processing system according to one embodiment of the present invention will now be described. In the information processing system of this embodiment, it is assumed that an avatar will converse with a user, and the content of the avatar's speech can be generated using, for example, a generation AI. In this embodiment, when generating a message (outgoing message) that the avatar will speak to the user, the message is generated while referring to past messages (past messages), and some of the past messages are intentionally forgotten. For example, the older the message, the more likely it is to be forgotten. However, messages that are likely to make a deep impression on the user are not forgotten.

[0011] 1 is a diagram showing an example of the overall configuration of an information processing system. The information processing system of this embodiment is configured to include a management server 2. The management server 2 is communicably connected to a user terminal 1 via a communication network. The communication network is, for example, the Internet, and is constructed using a public telephone line network, a mobile phone line network, a wireless communication path, Ethernet (registered trademark), or the like.

[0012] The user terminal 1 is a computer operated by a user, and may be, for example, a smartphone, a tablet computer, or a personal computer.

[0013] The management server 2 is a computer that performs processing related to avatars. The management server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0014] <Management Server> FIG. 2 is a diagram illustrating an example of the hardware configuration of the management server 2. Note that the illustrated configuration is an example, and other configurations may also be used. The management server 2 includes a CPU 201, memory 202, storage device 203, communication interface 204, input device 205, and output device 206. The storage device 203 stores various data and programs, and is, for example, a hard disk drive, solid state drive, or flash memory. The communication interface 204 is an interface for connecting to a communication network, and is, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, a wireless communication device for wireless communication, or a USB (Universal Serial Bus) connector or RS232C connector for serial communication. The input device 205 is, for example, a keyboard, mouse, touch panel, button, microphone, or the like for inputting data. The output device 206 is, for example, a display, printer, speaker, or the like for outputting data. Each functional unit of the management server 2 described below is realized by the CPU 201 reading a program stored in the storage device 203 into the memory 202 and executing it, and each storage unit of the management server 2 is realized as part of the storage area provided by the memory 202 and the storage device 203.

[0015] 3 is a diagram illustrating an example of the software configuration of the management server 2. The management server 2 includes a message storage unit 231, a message transmission / reception unit 211, a message generation unit 212, a summarization unit 213, and a feeling estimation unit 214.

[0016] <Storage Unit> The message storage unit 231 stores messages sent and received between the user and the avatar (hereinafter referred to as past messages) in association with the user. The messages are assumed to be text data, but may also include video, still images, audio, etc. For example, a video (including audio) of a conversation between the user and an avatar and text extracted from the video by voice recognition can be stored in the message storage unit 231 as past messages.

[0017] <Functional Unit> The message transmission / reception unit 211 transmits and receives messages to and from the user.

[0018] In this embodiment, an information processing system uses an avatar as an interface when sending and receiving messages to a user. The avatar is a humanoid character rendered using CG and capable of visually and aurally influencing the user. The avatar's appearance can be generated using 3DCG rendering technology based on pre-designed polygon models and textures. For example, multiple avatar models with different genders, age groups, roles, etc. can be prepared, allowing the user to select the optimal model based on their attributes and preferences. The avatar's facial expressions, gestures, mouth movements, etc. can be animated in real time in synchronization with the content of the dialogue output from the message generation unit 213. The avatar can be rendered on various devices, such as smartphones, tablets, and large displays, depending on the context of the dialogue. The message transmission / reception unit 211 performs a process of having the avatar speak a message (outputting the message as the avatar's speech). The message transmission / reception unit 211 controls the output of messages generated by the message generation unit 212, described below, as the avatar's speech.

[0019] Furthermore, the message transmitting / receiving unit 211 can acquire, as messages, voices generated by the user, video images of the user, and the like. The message transmitting / receiving unit 211 can receive, for example, voices and video images collected and captured by a camera or microphone provided in the user terminal 1 from the user terminal 1. The message transmitting / receiving unit 211 can also analyze the acquired voice and generate text data. The message transmitting / receiving unit 211 can register the acquired message (video, voice, text data, etc.) in the message storage unit 231 in association with the user and time information (for example, date and time, etc.).

[0020] The message generation unit 212 generates a message to be sent to the user (hereinafter referred to as a "transmitted message"). The message generation unit 212 generates the transmitted message while referring to past messages. The message generation unit 212 can generate the transmitted message by, for example, providing a prompt to the large-scale language model that includes a past message corresponding to the user who is the conversation partner and an instruction to generate a new transmitted message.

[0021] When generating a message to be sent, the message generator 212 does not refer to some of the past messages. For example, the message generator 212 can delete some of the past messages from the prompt (do not include them in the prompt).

[0022] When deleting past messages, the message generation unit 212 can select past messages older than a predetermined period of time as those to be deleted based on the time information of the past messages. For example, past messages older than one week can be selected for deletion. The message generation unit 212 can also randomly select and delete messages at a predetermined rate from the past messages to be deleted. For example, 50% of the past messages to be deleted can be randomly selected and deleted. In this case, the message generation unit 212 can set the deletion rate based on the time information of the past messages so that the older the message, the higher the probability of deletion. For example, it can be set so that past messages from one week ago are deleted with a 50% probability, past messages from two weeks ago are deleted with a 70% probability, and past messages from three weeks ago are deleted with a 90% probability. In this way, by appropriately setting the deletion criteria and deletion rate for past messages, the older the message, the higher the probability of deletion, thereby approximating the human memory mechanism.

[0023] When generating a message to be sent, the message generator 212 can include a summary of one or more past messages (hereinafter, a summary message) in the prompt. The message summary is performed by the summarizer 213, which will be described later.

[0024] When generating a message to be sent, the message generating unit 212 may not delete or summarize past messages that are memorable to the user. For example, the message generating unit 212 may not delete or summarize past messages in which the degree of the user's emotion estimated by the emotion estimation unit 214 (described later) is equal to or greater than a predetermined value.

[0025] The summarizing unit 213 creates a summary message by summarizing past messages.

[0026] The summarization unit 213 can perform summarization using a known method. For example, the summarization unit 213 can use TF-IDF, which is one of the extraction-type summarization methods. TF-IDF calculates the importance of each word by considering the term frequency of each word in past messages and the inverse document frequency of each word in all past messages. A summary is then generated by extracting sentences with high importance. The summarization unit 213 can also use LexRank, a method for extracting semantically important sentences. LexRank is a method for representing the similarity between sentences in past messages as a graph and extracting sentences with high centrality on the graph. The similarity between sentences can be calculated, for example, based on the co-occurrence frequency of words in the sentences or the similarity between word embeddings. In this way, the summarization unit 213 can select and use a method that can appropriately summarize the content of past messages.

[0027] The summarizing unit 213 can also use an abstract summarization method when generating a summary message. Unlike extraction summarization, abstract summarization is a method of generating new sentences based on an understanding of the content of past messages, rather than using sentences from past messages as they are. The summarizing unit 213 can achieve abstract summarization using deep learning models such as a seq2seq model or a transformer. These models can be pre-trained using large-scale text data to generate natural sentences that take context into consideration. The summarizing unit 213 can use the pre-trained model to train the model so that past messages are used as input and summary messages are used as output. This enables more flexible summarization of the content of past messages. The summarizing unit 213 can also generate a summary message by, for example, providing a prompt including an instruction to summarize past messages and the past messages to a large-scale language model.

[0028] The summarizing unit 213 can create a shorter summary message for older past messages. The summarizing unit 213 can generate a shorter summary message for older messages based on time information of the past messages. For example, the summarizing unit 213 can summarize a past message from one week ago in about 100 characters, a past message from two weeks ago in about 50 characters, and a past message from three weeks ago in about 20 characters. In this way, by generating a shorter summary message for older messages, it is possible to approximate the human memory mechanism. For example, the summarizing unit 213 can include a character count constraint in a prompt provided to the large-scale language model and set the number of characters so that the number of characters is shorter for older past messages.

[0029] The summarizing unit 213 can also adjust the length of the summary message depending on the content of the past message. For example, the summarizing unit 213 can generate a longer summary message for a past message that is determined to be highly important. The importance of a past message can be determined based on, for example, the presence or absence of a specific keyword in the message, the length of the message, the sender of the message, etc. On the other hand, a shorter summary message can be generated for a past message that is determined to be less important.

[0030] The emotion deduction unit 214 deduces the emotion of the user based on past messages. The emotion deduction unit 214 can deduce the type and degree of emotion based on past messages. The emotion deduction unit 214 can deduce the user's reaction, for example, from messages (text, video, audio, etc.) using a technique such as emotion analysis.

[0031] The emotion estimation unit 214 can infer the type and / or degree of emotion based on, for example, a video image and a first learning model. Six basic emotions, for example, joy, anger, sadness, fear, surprise, and disgust, can be defined as types of emotions. Furthermore, the degree of emotion can be expressed, for example, by a real number ranging from 0 to 1 for each emotion category. For example, if the degree of "joy" is 0.8, it can be interpreted that the user is feeling a relatively strong emotion of joy.

[0032] Specific emotion analysis techniques can use, for example, both acoustic features extracted from speech data and linguistic features extracted from text data. Examples of acoustic features include pitch, pitch fluctuation, volume, volume fluctuation, and the proportion of high-frequency components in speech. Examples of linguistic features include the frequency of appearance of words expressing emotions, the proportion of positive and negative expressions, and the syntactic structure of sentences. The emotion estimation unit 214 extracts these features and performs machine learning using pre-prepared emotion-labeled training data to construct an emotion classifier. For example, a supervised learning algorithm such as a support vector machine or a neural network can be used to train a model that predicts emotion categories from the features. Alternatively, the predicted probability for each category may be used as the degree of emotion.

[0033] To perform emotion analysis that takes context into account, the emotion estimation unit 214 may analyze time-series changes in emotions by referring to the history of past messages. For example, emotions can be analyzed for each segment of past messages separated by a certain time interval, and the transition of emotions between segments can be modeled using a Markov model or the like. This allows for a more detailed understanding of dynamic changes in the user's emotional state.

[0034] To perform emotion analysis that takes context into account, the emotion estimation unit 214 may analyze time-series changes in emotions by referring to the history of past messages. For example, emotions can be analyzed for each segment of past messages separated by a certain time interval, and the transition of emotions between segments can be modeled using a Markov model or the like. This allows for a more detailed understanding of dynamic changes in the user's emotional state.

[0035] <Operation> FIG. 4 is a diagram illustrating the operation of the management server 2.

[0036] The management server 2 acquires past messages sent and received between the user (S301) and registers the past messages in the message storage unit 231 (S302). The management server 2 reads past messages corresponding to the user (S303), deletes or summarizes old past messages, and includes them in a prompt for generating a message to be sent (S304), provides the prompt to a large-scale language model to generate a message to be sent (S305), and sends the generated message to be sent (having an avatar speak) (S306).

[0037] As described above, according to the information processing system of this embodiment, it is possible to carry out a conversation while referring to past messages exchanged between users and forgetting old messages.

[0038] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and equivalents thereof are also included in the present invention.

[0039] For example, the processing by each of the functional units of the management server 2 described above may be performed by any of the functional units. Also, a different functional unit that performs part of the processing by each of the functional units described above may be added. Also, the functional units of the management server 2 may be distributed across multiple computers.

[0040] Furthermore, the information stored in each storage unit of the management server 2 may be stored in any of the storage units. That is, the information stored in the above-mentioned multiple storage units may be stored in one storage unit, or part of the information stored in one of the above-mentioned storage units may be stored in another storage unit.

[0041] <Modification 1> In the above embodiment, the deletion rate of past messages and the length of summaries are set uniformly for all users, but this is not limiting. The deletion rate and the length of summaries may be adaptively changed for each user.

[0042] For example, the message generation unit 212 may gradually increase the deletion rate according to the accumulated amount of dialogue history with the user. Specifically, the message generation unit 212 can increase the deletion rate when the number of past messages with the user exceeds a predetermined threshold. For example, the deletion rate can be set to increase by 10% when the number of past messages with the user exceeds 1,000, and to increase by another 10% when the number of past messages with the user exceeds 2,000. In this way, the longer the dialogue with the user, the more actively older past messages are deleted, thereby making it possible to adaptively control the storage amount according to the user.

[0043] The message generator 212 may also adaptively change the deletion rate and the length of the summary based on the characteristics of the user's responses. For example, if the user's responses are mostly short sentences, the message generator 212 can increase the deletion rate and shorten the length of the summary. This is because it can be determined that the user prefers concise responses. On the other hand, if the user's responses are mostly long sentences, the message generator 212 can decrease the deletion rate and lengthen the length of the summary. This is because it can be determined that the user prefers detailed responses. In this way, by adapting to the characteristics of the user's responses, it is possible to realize a dialogue that suits the user's preferences.

[0044] Furthermore, the message generator 212 may adaptively change the deletion rate and the length of the summary based on the user's attribute information. For example, if the user is older, the message generator 212 may decrease the deletion rate and increase the length of the summary. This is because older users tend to have a poorer memory. On the other hand, if the user is younger, the message generator 212 may increase the deletion rate and shorten the length of the summary. This is because younger users tend to have a better memory. In this way, by taking the user's attribute information into consideration, it is possible to realize a dialogue that is tailored to the user's characteristics.

[0045] In the above embodiment, past messages are deleted or summarized uniformly regardless of the content of the message, but the present invention is not limited to this. The method of summarizing may be changed depending on the content of the deleted message.

[0046] For example, if a past message to be deleted is a response to a question from a user, the summarizing unit 213 can summarize the past message in detail without deleting it. Specifically, the summarizing unit 213 can determine the question from the user based on the presence or absence of interrogative words (e.g., "what," "where," "who," "when," "why," etc.) in the past message and the presence or absence of a symbol (e.g., "?") at the end of the sentence. Then, for a past message determined to be a response to a question from a user, the summarizing unit 213 can generate a summary message with a length that is 80% or more of the original past message without deleting it. In this way, because responses to questions from users are considered to be highly important in a conversation, detailed summarization can prevent information loss.

[0047] If a past message to be deleted contains small talk-like content, the summarizing unit 213 can actively delete the past message or only roughly summarize it. Specifically, the summarizing unit 213 can determine whether the past message contains small talk-like content based on the presence or absence of specific keywords (e.g., "weather," "mood," "small talk," etc.) in the past message. For past messages determined to contain small talk-like content, the summarizing unit 213 can set a higher deletion rate or limit the length of the summary message to approximately 20% of the original past message. Since small talk-like content is considered to be less important in a conversation, actively deleting or roughly summarizing the content can reduce unnecessary references by the message generating unit 212.

[0048] Furthermore, the summarizing unit 213 can extract important keywords from the past messages to be deleted and generate a summary message that includes these keywords. Specifically, the summarizing unit 213 can analyze the past messages using a morphological analyzer and extract nouns, proper nouns, unknown words, and the like as important keywords. The summarizing unit 213 can then generate a summary message that includes as many of the extracted keywords as possible. This allows important information contained in the past messages to be deleted to be reflected in the summary message.

[0049] <Modification 3>

[0050] The method of summarization may be changed based on the result of emotion estimation.

[0051] For example, the summarization unit 213 can change the length of the summary message based on the type of emotion estimated by the emotion estimation unit 214. Specifically, the summarization unit 213 can summarize in more detail a past message in which a positive emotion such as joy or surprise has been detected. For example, the summarization unit 213 can generate a summary message with a length that is 60% or more of the original past message in which a past message in which joy or surprise has been detected. In this way, past messages in which the user expressed positive emotions are considered to be of high importance in the conversation, and therefore, by summarizing them in detail, the user's emotions can be appropriately reflected.

[0052] The summarization unit 213 can make a rough summary of a past message in which a negative emotion such as sadness or anger is detected. For example, the summarization unit 213 can generate a summary message that is about 30% of the length of the original past message in which a sadness or anger is detected. In this way, since it is considered preferable that a user does not remember in detail past messages in which a negative emotion is expressed in a conversation, the impact of negative content can be minimized by making a rough summary.

[0053] The summarizing unit 213 can continuously change the length of the summary message based on the degree of emotion estimated by the emotion estimation unit 214. Specifically, the summarizing unit 213 can lengthen the summary message as the degree of emotion estimated from the past message increases. For example, the summarizing unit 213 can be configured to generate a summary message with a length that is 80% or more of the original past message for a past message with a joy level of 0.8 or higher, and to generate a summary message with a length that is 60% or more of the original past message for a past message with a joy level of 0.6 or higher but less than 0.8. In this way, by continuously changing the length of the summary message according to the degree of emotion, the user's emotions can be reflected in more detail.

[0054] The summarizing unit 213 can change the style of the summary message based on the type of emotion estimated by the emotion estimation unit 214. Specifically, the summarizing unit 213 can generate a summary message in a more casual style for past messages in which positive emotions such as joy or surprise are detected. For example, exclamatory expressions such as "Amazing!" or "Good job!" can be included. On the other hand, the summarizing unit 213 can generate a summary message in a more formal style for past messages in which negative emotions such as sadness or anger are detected. For example, formal expressions such as "That's unfortunate" or "I'm sorry" can be used. In this way, by changing the style of the summary message depending on the type of emotion, it is possible to realize a dialogue that is in tune with the user's emotions.

[0055] <Disclosures> The present disclosure also includes the following configurations. [Item 1] An information processing system comprising: a message storage unit that stores past messages, which are messages sent and received between a user and the user, in association with the user; and a message generation unit that generates the outgoing message by providing a large-scale language model with a prompt that includes the past message corresponding to the user who is a dialogue partner and an instruction to generate an outgoing message, which is a message to be newly sent to the user, wherein the message generation unit deletes a portion of the past message corresponding to the user who is a dialogue partner from the prompt. [Item 2] The information processing system according to Item 1, further comprising: a summarization unit that creates a summary message that summarizes the past message to be deleted, and the message generation unit includes the summary message in the prompt in place of the past message to be deleted. [Item 3] The information processing system according to Item 2, further comprising: a summarization unit that creates a summary message that is shorter the older the past message is. [Item 4] The information processing system according to Item 2, wherein the summarization unit generates the summarized message by providing a prompt including an instruction to summarize the past message and the past message to the large-scale language model. [Item 5] The information processing system according to Item 1, further comprising: an emotion estimation unit that estimates a degree of emotion of the user based on the first message, and the message generation unit does not delete any of the past messages whose degree of emotion is equal to or greater than a predetermined value.[Item 6] An information processing method comprising: a step of storing past messages, which are messages sent and received between a user and the user, in association with the user; and a step of generating the outgoing message by providing a large-scale language model with a prompt including the past messages corresponding to the user who is a conversation partner and an instruction to generate an outgoing message, which is a message to be newly sent to the user; wherein in the step of generating the outgoing message, the computer deletes a part of the past messages corresponding to the user who is a conversation partner from the prompt.

[0056] 1 User terminal 2 Management server

Claims

1. An information processing system comprising: a message storage unit that stores past messages, which are messages sent and received between a user and a user, in association with the user; and a message generation unit that generates the outgoing message by providing a large-scale language model with a prompt that includes the past message corresponding to the user who is a conversation partner and an instruction to generate an outgoing message, which is a message to be newly sent to the user; wherein the message generation unit deletes a portion of the past message corresponding to the user who is a conversation partner from the prompt.

2. An information processing system according to claim 1, further comprising a summarizing unit that creates a summary message summarizing the past message to be deleted, and the message generating unit includes the summary message in the prompt in place of the past message to be deleted.

3. An information processing system according to claim 2, wherein the summarizing unit creates a summary message that is shorter the older the past message.

4. An information processing system according to claim 2, wherein the summarizing unit generates the summary message by providing a prompt to the large-scale language model, the prompt including an instruction to summarize the past message and the past message.

5. An information processing system according to claim 1, comprising an emotion estimation unit that estimates the degree of emotion of the user based on the first message, and wherein the message generation unit does not delete past messages in which the degree of emotion is equal to or greater than a predetermined value.

6. An information processing method comprising: a step of storing past messages, which are messages sent and received between a user and the user, in association with the user; and a step of generating the outgoing message by providing a large-scale language model with a prompt including the past messages corresponding to the user who is a conversation partner and an instruction to generate an outgoing message, which is a message to be newly sent to the user; wherein in the step of generating the outgoing message, the computer deletes a portion of the past messages corresponding to the user who is a conversation partner from the prompt.

Images