Information processing system and information processing method

Abstract

To efficiently and effectively generate a message of a conversation.SOLUTION: An information processing system includes a message storage unit configured to store a past message in association with a user, the past message being a message transmitted and received to and from the user, and a message generation unit configured to generate a transmission message by giving a large-scale language model a prompt including a past message corresponding to a user who is a dialogue partner and an instruction to generate the transmission message, the transmission message being a message to be newly transmitted to the user, wherein the message generation unit deletes a part of the past message corresponding to the user who is the dialogue partner from the prompt.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system and an information processing method. [Background technology]

[0002] A conversation program using artificial intelligence is available (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-531994 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a need to ensure efficient processing while maintaining conversational consistency.

[0005] The present invention has been made in view of the above background, and has an object to provide a technique that can generate conversational messages efficiently and effectively. [Means for solving the problem]

[0006] The main invention of the present invention for solving the above problem is an information processing system comprising: a message storage unit that stores past messages, which are messages sent and received between a user and a user, in association with the user; and a message generation unit that generates the outgoing message by providing a prompt to a large-scale language model, the prompt including the past message corresponding to the user who is a dialogue partner and an instruction to generate an outgoing message, which is a message to be newly sent to the user, and is characterized in that the message generation unit deletes a portion of the past message corresponding to the user who is a dialogue partner from the prompt.

[0007] Other problems and solutions disclosed in this application will be made clear in the section on preferred embodiments of the invention and the drawings. [Effects of the Invention]

[0008] According to the present invention, conversational messages can be generated efficiently and effectively. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of an information processing system. [Figure 2] FIG. 2 illustrates an example of a hardware configuration of a management server 2. [Figure 3] FIG. 2 illustrates an example of the software configuration of a management server 2. [Figure 4] FIG. 10 is a diagram illustrating the operation of the management server 2. DETAILED DESCRIPTION OF THE INVENTION

[0010] <System Overview> An information processing system according to one embodiment of the present invention will be described below. In the information processing system of this embodiment, it is assumed that an avatar will converse with a user, and the content of the avatar's speech can be generated using, for example, a generation AI. In this embodiment, when generating a message (outgoing message) that the avatar will speak to the user, the message is generated while referring to past messages (past messages), and some of the past messages are intentionally forgotten. For example, the older the message, the more likely it is to forget it. However, messages that are likely to leave a deep impression on the user will not be forgotten.

[0011] 1 is a diagram showing an example of the overall configuration of an information processing system. The information processing system of this embodiment is configured to include a management server 2. The management server 2 is communicably connected to a user terminal 1 via a communication network. The communication network is, for example, the Internet, and is constructed using a public telephone line network, a mobile phone line network, a wireless communication path, Ethernet (registered trademark), etc.

[0012] The user terminal 1 is a computer operated by a user, and may be, for example, a smartphone, a tablet computer, or a personal computer.

[0013] The management server 2 is a computer that performs processing related to avatars. The management server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0014] <Administration Server> FIG. 2 is a diagram illustrating an example of the hardware configuration of the management server 2. Note that the illustrated configuration is an example, and other configurations may also be used. The management server 2 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 stores various data and programs, and is, for example, a hard disk drive, a solid state drive, or a flash memory. The communication interface 204 is an interface for connecting to a communication network, and is, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, a wireless communication device for wireless communication, or a USB (Universal Serial Bus) connector or an RS232C connector for serial communication. The input device 205 is used to input data, and is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, or the like. The output device 206 is used to output data, and is, for example, a display, a printer, a speaker, or the like. Each functional unit of the management server 2 described below is realized by the CPU 201 reading a program stored in the storage device 203 into the memory 202 and executing it, and each storage unit of the management server 2 is realized as part of the storage area provided by the memory 202 and the storage device 203.

[0015] 3 is a diagram illustrating an example of the software configuration of the management server 2. The management server 2 includes a message storage unit 231, a message transmission / reception unit 211, a message generation unit 212, a summarization unit 213, and a feeling estimation unit 214.

[0016] <Storage section> The message storage unit 231 stores messages sent and received between the user and the user (hereinafter referred to as past messages) in association with the user. The messages are assumed to be text data, but may also include video, still images, audio, etc. For example, a video (including audio) of a conversation between the user and an avatar and text extracted from the video by voice recognition can be stored in the message storage unit 231 as past messages.

[0017] <Functional section> The message transmitting / receiving unit 211 transmits and receives messages to and from the user.

[0018] In this embodiment, an information processing system uses an avatar as an interface when sending and receiving messages to a user. The avatar is a humanoid character rendered using CG and capable of visually and aurally influencing the user. The avatar's appearance can be generated using 3DCG rendering technology based on a pre-designed polygon model and texture. For example, multiple avatar models with different genders, age groups, roles, etc. can be prepared, allowing the user to select the optimal model based on their attributes and preferences. The avatar's facial expressions, gestures, mouth movements, etc. can be animated in real time in synchronization with the content of the dialogue output from the message generation unit 213. The avatar can be rendered on various devices, such as smartphones, tablets, and large displays, depending on the context of the dialogue. The message transmission / reception unit 211 performs processing to have the avatar speak a message (output the message as the avatar's speech). The message transmission / reception unit 211 controls the output of messages generated by the message generation unit 212 (described later) as the avatar's speech.

[0019] Furthermore, the message transmitting / receiving unit 211 can acquire, as a message, voice generated by the user, video images of the user's state, and the like. For example, the message transmitting / receiving unit 211 can receive from the user terminal 1 voice and video images collected and captured by a camera or microphone provided in the user terminal 1. The message transmitting / receiving unit 211 can also analyze the acquired voice and generate text data. The message transmitting / receiving unit 211 can register the acquired message (video, voice, text data, etc.) in the message storage unit 231 in association with the user and time information (for example, date and time, etc.).

[0020] The message generating unit 212 generates a message to be sent to the user (hereinafter, a "sent message"). The message generator 212 generates a message to be sent while referring to past messages. For example, the message generator 212 can generate a message to be sent by providing a prompt to the large-scale language model, the prompt including a past message corresponding to the user who is the conversation partner and an instruction to generate a new message to be sent.

[0021] When generating a message to be sent, the message generator 212 does not refer to some of the past messages. For example, the message generator 212 can delete some of the past messages from the prompt (do not include them in the prompt).

[0022] When deleting past messages, the message generation unit 212 can select past messages older than a predetermined period of time as those to be deleted based on the time information of the past messages. For example, past messages older than one week can be selected for deletion. Furthermore, the message generation unit 212 can randomly select and delete messages at a predetermined rate from the past messages to be deleted. For example, 50% of the past messages to be deleted can be randomly selected and deleted. In this case, the message generation unit 212 can set the deletion rate based on the time information of the past messages so that the older the message, the higher the probability of deletion. For example, it can be set so that past messages from one week ago are deleted with a 50% probability, past messages from two weeks ago are deleted with a 70% probability, and past messages from three weeks ago are deleted with a 90% probability. In this way, by appropriately setting the deletion criteria and deletion rate for past messages, the older the message, the higher the probability of deletion, making it possible to approximate the human memory mechanism.

[0023] When generating a message to be sent, the message generator 212 can include a summary of one or more past messages (hereinafter, a summary message) in the prompt. The message summary is performed by the summarizing unit 213, which will be described later.

[0024] When generating a message to be sent, the message generation unit 212 may not delete or summarize past messages that are memorable to the user. For example, the message generation unit 212 may not delete or summarize past messages in which the degree of the user's emotion estimated by the emotion estimation unit 214 (described later) is equal to or greater than a predetermined value.

[0025] The summarizing unit 213 creates a summary message by summarizing past messages.

[0026] The summarization unit 213 can perform summarization using a known method. For example, the summarization unit 213 can use TF-IDF, which is one of the extraction-type summarization methods. TF-IDF calculates the importance of each word by considering the term frequency of each word in the past message and the inverse document frequency of each word in all the past messages. Then, a summary is generated by extracting sentences with high importance. The summarization unit 213 can also use LexRank, which is a method for extracting semantically important sentences. LexRank is a method for representing the similarity between sentences in the past message as a graph and extracting sentences with high centrality on the graph. The similarity between sentences can be calculated, for example, by the co-occurrence frequency of words in the sentence or the similarity between the distributed representations of words. In this way, the summarization unit 213 can select and use a method that can appropriately summarize the contents of the past message.

[0027] The summarization unit 213 can also use an abstract summarization technique when generating a summary message. Unlike extraction summarization, abstract summarization is a technique that does not use sentences in past messages as they are, but rather generates new sentences after understanding the content of the past messages. The summarization unit 213 can realize abstract summarization using deep learning models such as a seq2seq model or a transformer. These models can be pre-trained using large-scale text data to generate natural sentences that take context into consideration. The summarization unit 213 can use the pre-trained model to train the model so that past messages are used as input and summary messages are used as output. This enables more flexible summarization of the contents of past messages. The summarization unit 213 can also generate a summary message by, for example, providing a prompt including an instruction to summarize past messages and the past messages to a large-scale language model.

[0028] The summarizing unit 213 can create a shorter summary message for older past messages. The summarizing unit 213 can generate a shorter summary message for older messages based on time information of the past messages. For example, the summarizing unit 213 can summarize a past message from one week ago in about 100 characters, a past message from two weeks ago in about 50 characters, and a past message from three weeks ago in about 20 characters. In this way, by generating a shorter summary message for older messages, it is possible to approximate the human memory mechanism. For example, the summarizing unit 213 can include a character count constraint in the prompt provided to the large-scale language model and set the character count so that the number of characters is shorter for older past messages.

[0029] The summarizing unit 213 can also adjust the length of the summary message depending on the content of the past message. For example, the summarizing unit 213 can generate a longer summary message for a past message that is determined to be highly important. The importance of a past message can be determined based on, for example, the presence or absence of a specific keyword in the message, the length of the message, the sender of the message, etc. On the other hand, a shorter summary message can be generated for a past message that is determined to be less important.

[0030] The emotion estimation unit 214 estimates the emotion of the user based on past messages. The emotion estimation unit 214 can estimate the type and degree of emotion based on past messages. The emotion estimation unit 214 can estimate the user's reaction, for example, from messages (text, video, audio, etc.) using techniques such as emotion analysis.

[0031] The emotion estimation unit 214 can infer the type and / or degree of emotion based on, for example, a video and a first learning model. As the types of emotions, for example, six basic emotions can be defined: joy, anger, sadness, fear, surprise, and disgust. Furthermore, the degree of emotion can be expressed, for example, by a real number ranging from 0 to 1 for each emotion category. For example, if the degree of "joy" is 0.8, it can be interpreted that the user is feeling a relatively strong emotion of joy.

[0032] Specific emotion analysis techniques can use both acoustic features extracted from speech data and linguistic features extracted from text data. Examples of acoustic features include pitch, pitch variance, volume, volume variance, and the proportion of high-frequency components in speech. Examples of linguistic features include the frequency of emotion-expressing words, the proportion of positive and negative expressions, and the syntactic structure of sentences. The emotion estimation unit 214 extracts these features and performs machine learning using pre-prepared emotion-labeled training data to construct an emotion classifier. For example, a supervised learning algorithm such as a support vector machine or a neural network can be used to train a model that predicts emotion categories from the features. Alternatively, the predicted probability for each category may be used as the degree of emotion.

[0033] To perform emotion analysis that takes context into account, the emotion estimation unit 214 may analyze time-series changes in emotions by referring to the history of past messages. For example, emotions can be analyzed for each segment of past messages separated by a fixed time period, and the transition of emotions between segments can be modeled using a Markov model or the like. This allows for a more detailed understanding of dynamic changes in the user's emotional state.

[0034] To perform emotion analysis that takes context into account, the emotion estimation unit 214 may analyze time-series changes in emotions by referring to the history of past messages. For example, emotions can be analyzed for each segment of past messages separated by a fixed time period, and the transition of emotions between segments can be modeled using a Markov model or the like. This allows for a more detailed understanding of dynamic changes in the user's emotional state.

[0035] <Operation> FIG. 4 is a diagram illustrating the operation of the management server 2.

[0036] The management server 2 acquires past messages sent and received between the user (S301) and registers the past messages in the message storage unit 231 (S302). The management server 2 reads past messages corresponding to the user (S303), deletes or summarizes old past messages, and includes them in a prompt for generating a message to be sent (S304), provides the prompt to a large-scale language model to generate a message to be sent (S305), and sends the generated message to be sent (having an avatar speak it) (S306).

[0037] As described above, according to the information processing system of this embodiment, it is possible to carry out a conversation while referring to past messages exchanged between users and forgetting old messages.

[0038] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and equivalents thereof are also included in the present invention.

[0039] For example, the processing by each of the functional units of the management server 2 described above may be performed by any of the functional units. Also, a different functional unit that performs part of the processing by each of the functional units described above may be added. Also, the functional units of the management server 2 may be distributed across multiple computers.

[0040] Furthermore, the information stored in each storage unit of the management server 2 may be stored in any of the storage units. That is, the information stored in the above-mentioned multiple storage units may be stored in one storage unit, or part of the information stored in one of the above-mentioned storage units may be stored in another storage unit.

[0041] <Variation 1> In the above embodiment, the deletion rate of past messages and the length of summaries are set uniformly for all users, but this is not limiting. The deletion rate and the length of summaries may be adaptively changed for each user.

[0042] For example, the message generation unit 212 may gradually increase the deletion rate according to the accumulated amount of dialogue history with the user. Specifically, the message generation unit 212 can increase the deletion rate when the number of past messages with the user exceeds a predetermined threshold. For example, the deletion rate can be set to increase by 10% when the number of past messages with the user exceeds 1,000, and to increase by another 10% when the number exceeds 2,000. In this way, the longer the dialogue with the user, the more actively older past messages are deleted, making it possible to adaptively control the storage amount according to the user.

[0043] Furthermore, the message generator 212 may adaptively change the deletion rate and the length of the summary based on the characteristics of the user's responses. For example, if the user's responses are mostly short sentences, the message generator 212 can increase the deletion rate and shorten the length of the summary. This is because it can be determined that the user prefers concise responses. On the other hand, if the user's responses are mostly long sentences, the message generator 212 can decrease the deletion rate and lengthen the length of the summary. This is because it can be determined that the user prefers detailed responses. In this way, by adapting to the characteristics of the user's responses, it is possible to realize a dialogue that suits the user's preferences.

[0044] Furthermore, the message generation unit 212 may adaptively change the deletion rate and the length of the summary based on the user's attribute information. For example, if the user is older, the message generation unit 212 can decrease the deletion rate and increase the length of the summary. This is because older users tend to have a poorer memory. On the other hand, if the user is younger, the deletion rate can be increased and the length of the summary can be shortened. This is because younger users tend to have a better memory. In this way, by taking the user's attribute information into consideration, it is possible to realize a dialogue that is tailored to the user's characteristics.

[0045] <Variation 2> In the above embodiment, an example was shown in which past messages were deleted and summarized uniformly regardless of the content of the messages, but this is not limiting. The method of summarizing may be changed depending on the content of the deleted messages.

[0046] For example, if a past message to be deleted is a reply to a question from a user, the summarizing unit 213 can summarize the past message in detail without deleting it. Specifically, the summarizing unit 213 can determine the question from the user based on the presence or absence of interrogative words (e.g., "what," "where," "who," "when," "why," etc.) in the past message and the presence or absence of a symbol (e.g., "?") at the end of the sentence. Then, for a past message determined to be a reply to a question from a user, the summarizing unit 213 can generate a summary message with a length that is 80% or more of the original past message without deleting it. In this way, since answers to questions from users are considered to be highly important in a dialogue, missing information can be prevented by summarizing the past message in detail.

[0047] If a past message to be deleted has small talk-like content, the summarizing unit 213 can actively delete the past message or only roughly summarize it. Specifically, the summarizing unit 213 can determine whether the past message has small talk-like content based on the presence or absence of specific keywords (e.g., "weather," "mood," "small talk," etc.) in the past message. Then, for past messages determined to have small talk-like content, the summarizing unit 213 can set a higher deletion rate or limit the length of the summary message to about 20% of the original past message. In this way, since small talk-like content is considered to be less important in a conversation, unnecessary references by the message generating unit 212 can be reduced by actively deleting or roughly summarizing the small talk-like content.

[0048] Furthermore, the summarization unit 213 can extract important keywords from the past messages to be deleted and generate a summary message that includes these keywords. Specifically, the summarization unit 213 can analyze the past messages using a morphological analyzer and extract nouns, proper nouns, unknown words, and the like as important keywords. The summarization unit 213 can then generate a summary message that includes as many of the extracted keywords as possible. This allows important information contained in the past messages to be deleted to be reflected in the summary message.

[0049] <Variation 3>

[0050] The method of summarization may be changed based on the result of emotion estimation.

[0051] For example, the summarization unit 213 can change the length of the summary message based on the type of emotion estimated by the emotion estimation unit 214. Specifically, the summarization unit 213 can summarize in more detail past messages in which positive emotions such as joy or surprise have been detected. For example, the summarization unit 213 can generate a summary message with a length that is 60% or more of the original past message in terms of past messages in which emotions such as joy or surprise have been detected. In this way, past messages in which the user has expressed positive emotions are considered to be of high importance in a conversation, and therefore, by summarizing them in detail, the user's emotions can be appropriately reflected.

[0052] The summarization unit 213 can make a rough summary of a past message in which a negative emotion such as sadness or anger is detected. For example, the summarization unit 213 can generate a summary message that is about 30% of the length of the original past message in which a sadness or anger is detected. In this way, since it is considered preferable that a user does not remember in detail past messages in which a negative emotion is expressed in a conversation, the impact of negative content can be minimized by making a rough summary.

[0053] The summarizing unit 213 can continuously change the length of the summary message based on the degree of emotion estimated by the emotion estimation unit 214. Specifically, the summarizing unit 213 can lengthen the summary message as the degree of emotion estimated from the past message increases. For example, the summarizing unit 213 can be configured to generate a summary message that is 80% or more the length of the original past message for past messages with a joy level of 0.8 or higher, and to generate a summary message that is 60% or more the length of the original past message for past messages with a joy level of 0.6 or higher but less than 0.8. In this way, by continuously changing the length of the summary message according to the degree of emotion, the user's emotions can be reflected in more detail.

[0054] The summarizing unit 213 can change the style of the summary message based on the type of emotion estimated by the emotion estimation unit 214. Specifically, the summarizing unit 213 can generate a summary message in a more casual style for past messages in which positive emotions such as joy or surprise are detected. For example, exclamatory expressions such as "Amazing!" or "Good job!" can be included. On the other hand, the summarizing unit 213 can generate a summary message in a more formal style for past messages in which negative emotions such as sadness or anger are detected. For example, formal expressions such as "That's unfortunate" or "I'm sorry" can be used. In this way, by changing the style of the summary message depending on the type of emotion, it is possible to realize a dialogue that is in tune with the user's emotions.

[0055] <Disclosures> The present disclosure also includes the following configurations. [Item 1] a message storage unit that stores past messages that are messages transmitted and received between the user and the message storage unit, in association with the user; a message generation unit that generates the transmission message by providing a large-scale language model with a prompt including the past message corresponding to the user who is a conversation partner and an instruction to generate a transmission message that is a message to be newly transmitted to the user; Equipped with the message generation unit deletes a portion of the past message corresponding to the conversation partner user from the prompt; An information processing system characterized by: [Item 2] Item 1, an information processing system according to item 1, a summarizing unit that creates a summary message summarizing the past messages to be deleted; the message generator includes the summary message in the prompt in place of the deleted past message; An information processing system characterized by: [Item 3] Item 2. The information processing system according to item 2, the summarizing unit creates a summary message that is shorter the older the past message is; An information processing system characterized by: [Item 4] Item 2. The information processing system according to item 2, the summarizing unit provides a prompt including an instruction to summarize the past message and the past message to the large-scale language model to generate the summarized message; An information processing system characterized by: [Item 5] Item 1, an information processing system according to item 1, an emotion estimation unit that estimates a degree of emotion of the user based on the first message; the message generation unit does not delete the past messages in which the degree of emotion is equal to or greater than a predetermined value; An information processing system characterized by: [Item 6] a step of storing past messages, which are messages transmitted and received between the user and the user, in association with the user; generating a message to be sent by providing a prompt to a large-scale language model, the prompt including the past message corresponding to the user who is a conversation partner and an instruction to generate a new message to be sent to the user; The computer executes In the step of generating the outgoing message, the computer deletes a portion of the past message corresponding to the user who is the other party from the prompt; An information processing method comprising: [Explanation of symbols]

[0056] 1. User terminal 2 Management Server

Claims

1. a message storage unit that stores past messages that are messages transmitted and received between the user and the message storage unit, in association with the user; a message generation unit that generates the transmission message by providing a large-scale language model with a prompt including the past message corresponding to the user who is a conversation partner and an instruction to generate a transmission message that is a message to be newly transmitted to the user; Equipped with the message generation unit deletes a portion of the past message corresponding to the conversation partner user from the prompt; An information processing system characterized by:

2. 2. The information processing system according to claim 1, a summarizing unit that creates a summary message summarizing the past messages to be deleted; the message generator includes the summary message in the prompt in place of the deleted past message; An information processing system characterized by:

3. 3. The information processing system according to claim 2, the summarizing unit creates a summary message that is shorter the older the past message is; An information processing system characterized by:

4. 3. The information processing system according to claim 2, the summarizing unit provides a prompt including an instruction to summarize the past message and the past message to the large-scale language model to generate the summarized message; An information processing system characterized by:

5. 2. The information processing system according to claim 1, an emotion estimation unit that estimates a degree of emotion of the user based on the first message; the message generation unit does not delete the past messages in which the degree of emotion is equal to or greater than a predetermined value; An information processing system characterized by:

6. a step of storing past messages, which are messages transmitted and received between the user and the user, in association with the user; generating a message to be sent by providing a prompt to a large-scale language model, the prompt including the past message corresponding to the user who is a conversation partner and an instruction to generate a new message to be sent to the user; The computer executes In the step of generating the outgoing message, the computer deletes a portion of the past message corresponding to the user who is the other party from the prompt; An information processing method comprising:

Images