Information processing system and information processing method

Abstract

To enable efficient and effective generation of conversation messages. [Solution] An information processing system comprising a message storage unit that stores past messages, which are messages sent and received between a user, in association with the user, and a message generation unit that generates a transmission message by providing a large-scale language model with a prompt that includes the past message corresponding to the other user and an instruction to generate a transmission message, which is a new message to be sent to the user, and is characterized in that the message generation unit deletes a portion of the past message corresponding to the other user from the prompt.

Description

[Technical field]

[0001] The present invention relates to an information processing system and an information processing method. [Background technology]

[0002] A conversation program using artificial intelligence has been provided (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-531994 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a demand for efficient processing while maintaining the consistency of conversation.

[0005] The present invention has been made in view of the above background, and has an object to provide a technique capable of generating conversational messages efficiently and effectively. [Means for solving the problem]

[0006] The main invention of the present invention for solving the above problem is an information processing system comprising a message storage unit that stores past messages, which are messages sent and received between a user, in correspondence with the user, and a message generation unit that generates the outgoing message by providing a prompt to a large-scale language model that includes the past messages corresponding to the user who is a dialogue partner and an instruction to generate an outgoing message, which is a message to be newly sent to the user, and is characterized in that the message generation unit deletes a portion of the past messages corresponding to the user who is a dialogue partner from the prompt.

[0007] Other problems and solutions disclosed in this application will be made clear in the description of the preferred embodiments of the invention and the drawings. Effect of the Invention

[0008] According to the present invention, conversational messages can be generated efficiently and effectively. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an overall configuration of an information processing system. [Diagram 2] 2 is a diagram illustrating an example of a hardware configuration of a management server 2. FIG. [Diagram 3] 2 illustrates an example of the software configuration of a management server 2. FIG. [Figure 4] FIG. 13 is a diagram illustrating the operation of the management server 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <System Overview> An information processing system according to one embodiment of the present invention will be described below. In the information processing system of this embodiment, it is assumed that an avatar will converse with a user, and the content of the avatar's speech can be generated, for example, using a generation AI. In this embodiment, when generating a message (outgoing message) that the avatar will speak to the user, the message is generated while referring to past messages (past messages), and some of the past messages are intentionally forgotten. For example, the older the message, the more likely it is that the message will be forgotten. However, messages that are likely to leave a deep impression on the user will not be forgotten.

[0011] 1 is a diagram showing an example of the overall configuration of an information processing system. The information processing system of this embodiment is configured to include a management server 2. The management server 2 is communicably connected to a user terminal 1 via a communication network. The communication network is, for example, the Internet, and is constructed by a public telephone line network, a mobile phone line network, a wireless communication path, Ethernet (registered trademark), etc.

[0012] The user terminal 1 is a computer operated by a user. The user terminal 1 may be, for example, a smartphone, a tablet computer, or a personal computer.

[0013] The management server 2 is a computer that performs processing related to avatars. The management server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0014] <Management Server> FIG. 2 is a diagram showing an example of the hardware configuration of the management server 2. Note that the illustrated configuration is an example, and other configurations may be used. The management server 2 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 is, for example, a hard disk drive, a solid state drive, or a flash memory that stores various data and programs. The communication interface 204 is an interface for connecting to a communication network, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or an RS232C connector for serial communication, etc. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, etc. that input data. The output device 206 is, for example, a display, a printer, a speaker, etc. that output data. Each functional unit of the management server 2 described below is realized by the CPU 201 reading a program stored in the storage device 203 into the memory 202 and executing it, and each storage unit of the management server 2 is realized as part of the storage area provided by the memory 202 and the storage device 203.

[0015] 3 is a diagram illustrating an example of the software configuration of the management server 2. The management server 2 includes a message storage unit 231, a message transmission / reception unit 211, a message generation unit 212, a summary unit 213, and a feeling estimation unit 214.

[0016] <Storage section> The message storage unit 231 stores messages (hereinafter, past messages) transmitted and received between the user and the user in association with the user. The messages are assumed to be text data, but may also include video, still images, audio, and the like. For example, a video (including audio) of a conversation between the user and an avatar and text extracted from the video by voice recognition can be stored in the message storage unit 231 as past messages.

[0017] <Functional section> The message transmitting / receiving unit 211 transmits and receives messages to and from the user.

[0018] In this embodiment, an avatar is used as an interface when the information processing system transmits and receives messages to a user. The avatar is a human-shaped character drawn by CG, and can act visually and audibly on the user. The appearance of the avatar can be generated by using a 3DCG rendering technique based on a pre-designed polygon model and texture. For example, a plurality of avatar models with different genders, age groups, roles, etc. can be prepared, and the most suitable model can be selected according to the attributes and preferences of the user. The facial expressions, gestures, mouth movements, etc. of the avatar can be animated and controlled in real time in synchronization with the contents of the dialogue output from the message generation unit 213. The avatar can be drawn on various devices such as a smartphone, tablet, and large display according to the context of the dialogue. The message transmission / reception unit 211 performs a process of making the avatar speak a message (outputting the message as the avatar's speech). The message transmission / reception unit 211 controls the output of a message generated by the message generation unit 212, which will be described later, as the avatar's speech.

[0019] Furthermore, the message transmitting / receiving unit 211 can acquire, as a message, voice generated by the user, video images capturing the user's state, and the like. For example, the message transmitting / receiving unit 211 can receive from the user terminal 1 voice and video images collected and captured by a camera or microphone provided in the user terminal 1. The message transmitting / receiving unit 211 can also analyze the acquired voice and generate text data. The message transmitting / receiving unit 211 can register the acquired message (video, voice, text data, etc.) in the message storage unit 231 in association with the user and time information (for example, date and time, etc.).

[0020] The message generating unit 212 generates a message to be sent to a user (hereinafter, a "transmitted message"). The message generator 212 generates an outgoing message while referring to past messages. For example, the message generator 212 can generate an outgoing message by providing a prompt including a past message corresponding to a conversation partner user and an instruction to generate a new outgoing message to the large-scale language model.

[0021] When generating a message to be sent, the message generator 212 does not refer to some of the past messages. For example, the message generator 212 can delete some of the past messages from the prompt (do not include them in the prompt).

[0022] In deleting past messages, the message generating unit 212 can select past messages older than a predetermined period as the deletion target based on the time information of the past messages. For example, past messages older than one week can be the deletion target. Furthermore, the message generating unit 212 can randomly select and delete messages at a predetermined rate from the past messages to be deleted. For example, 50% of the past messages to be deleted can be randomly selected and deleted. In this case, the message generating unit 212 can set the deletion rate based on the time information of the past messages so that the older the message, the higher the probability of deletion. For example, it can be set so that the past messages one week ago are deleted with a 50% probability, the past messages two weeks ago are deleted with a 70% probability, and the past messages three weeks ago are deleted with a 90% probability. In this way, by appropriately setting the deletion criteria and deletion rate of past messages, the older the message, the higher the probability of deletion, making it possible to approach the human memory mechanism.

[0023] When generating a message to be sent, the message generator 212 can include a summary of one or more past messages (hereinafter, a summary message) in the prompt. The message is summarized by the summarizing unit 213, which will be described later.

[0024] When generating a message to be sent, the message generating unit 212 can avoid deleting or summarizing a past message that is memorable to the user. For example, the message generating unit 212 can avoid deleting or summarizing a past message in which the degree of the user's emotion estimated by the emotion estimation unit 214 described later is equal to or greater than a predetermined value.

[0025] The summarizing unit 213 creates a summary message by summarizing past messages.

[0026] The summarization unit 213 can perform summarization using a known method. For example, the summarization unit 213 can use TF-IDF, which is one of the extraction-type summarization methods. TF-IDF calculates the importance of each word by considering the occurrence frequency of each word in the past message (Term Frequency) and the inverse of the occurrence frequency of each word in all the past messages (Inverse Document Frequency). Then, a summary is generated by extracting sentences with high importance. The summarization unit 213 can also use LexRank, which is a method for extracting semantically important sentences. LexRank is a method for expressing the similarity between sentences in the past message in a graph and extracting sentences with high centrality on the graph. The similarity between sentences can be calculated, for example, by the co-occurrence frequency of words in the sentence or the similarity of distributed representations of words. In this way, the summarization unit 213 can select and use a method that can appropriately summarize the contents of the past message.

[0027] The summarization unit 213 can also use an abstract summarization method when generating a summary message. Unlike the extraction summarization, the abstract summarization is a method of generating new sentences after understanding the contents of the past messages, rather than using sentences in the past messages as they are. The summarization unit 213 can realize the abstract summarization using deep learning models such as the seq2seq model and the transformer. These models can generate natural sentences that take context into consideration by pre-training using large-scale text data. The summarization unit 213 can learn using a pre-trained model to input past messages and output summary messages. This makes it possible to more flexibly summarize the contents of past messages. The summarization unit 213 can also generate a summary message by giving a prompt including an instruction to summarize the past messages and the past messages to the large-scale language model, for example.

[0028] The summarizing unit 213 can create a summary message that is shorter the older the past message. The summarizing unit 213 can generate a summary message that is shorter the older the message, based on the time information of the past message. For example, the summarizing unit 213 can summarize a past message from one week ago in about 100 characters, a past message from two weeks ago in about 50 characters, and a past message from three weeks ago in about 20 characters. In this way, by generating a summary message that is shorter the older the message, it is possible to approach the human memory mechanism. For example, the summarizing unit 213 can include a constraint on the number of characters in a prompt given to the large-scale language model, and set the number of characters so that the number of characters is smaller for older past messages.

[0029] Furthermore, the summarizing unit 213 can adjust the length of the summary message depending on the contents of the past message. For example, the summarizing unit 213 can generate a longer summary message for a past message that is determined to be highly important. The importance of a past message can be determined based on, for example, the presence or absence of a specific keyword in the message, the length of the message, the sender of the message, etc. On the other hand, a shorter summary message can be generated for a past message that is determined to be less important.

[0030] The emotion deduction unit 214 deduces the emotion of the user based on the past messages. The emotion deduction unit 214 can deduces the type and degree of emotion based on the past messages. The emotion deduction unit 214 can deduces the reaction of the user by, for example, a method such as emotion analysis from the message (text, video, audio, etc.).

[0031] The emotion estimation unit 214 can infer the type of emotion and / or the degree of emotion based on, for example, a video image and a first learning model. As the types of emotions, for example, six basic emotions, namely, joy, anger, sadness, fear, surprise, and disgust, can be defined. Furthermore, the degree of emotion can be expressed, for example, by a real value ranging from 0 to 1 for each emotion category. For example, when the degree of "joy" is 0.8, it can be interpreted that the user is feeling a relatively strong emotion of joy.

[0032] As a specific method of emotion analysis, for example, both acoustic features extracted from voice data and linguistic features extracted from text data can be used. Examples of acoustic features include pitch, pitch fluctuation, volume, volume fluctuation, and the proportion of high-frequency components of the voice. Examples of linguistic features include the frequency of appearance of words expressing emotions, the proportion of positive and negative expressions, and the syntactic structure of sentences. The emotion estimation unit 214 can extract these features and build an emotion classifier by performing machine learning using learning data with emotion labels prepared in advance. For example, a model that predicts emotion categories from features can be trained using a supervised learning algorithm such as a support vector machine or a neural network. In addition, the predicted probability for each category may be used as the degree of emotion.

[0033] The emotion estimation unit 214 may analyze the time series change of emotion by referring to the history of past messages in order to perform emotion analysis taking into account the context. For example, the emotion can be analyzed for each segment of past messages separated by a certain time period, and the emotion transition between the segments can be modeled using a Markov model or the like. This allows the dynamic change of the user's emotional state to be grasped in more detail.

[0034] The emotion estimation unit 214 may analyze the time series change of emotion by referring to the history of past messages in order to perform emotion analysis taking into account the context. For example, the emotion can be analyzed for each segment of past messages separated by a certain time period, and the emotion transition between the segments can be modeled using a Markov model or the like. This allows the dynamic change of the user's emotional state to be grasped in more detail.

[0035] <Operation> FIG. 4 is a diagram illustrating the operation of the management server 2. As shown in FIG.

[0036] The management server 2 acquires past messages transmitted and received between the user (S301), and registers the past messages in the message storage unit 231 (S302). The management server 2 reads past messages corresponding to the user (S303), deletes or summarizes old past messages, and includes them in a prompt for generating a transmission message (S304), provides the prompt to a large-scale language model to generate a transmission message (S305), and transmits the generated transmission message (has an avatar speak it) (S306).

[0037] As described above, according to the information processing system of this embodiment, it is possible to carry out a conversation while referring to past messages exchanged between users and forgetting old messages.

[0038] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention, and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents thereof are also included in the present invention.

[0039] For example, the processes performed by each of the functional units of the management server 2 described above may be executed by any of the functional units. Also, a different functional unit that executes part of the processes performed by each of the functional units described above may be added. Also, the functional units of the management server 2 may be provided in a distributed manner on multiple computers.

[0040] Furthermore, the information stored in each storage unit of the management server 2 may be stored by any storage unit. That is, the information stored in the above-mentioned multiple storage units may be stored by one storage unit, or part of the information stored in one of the above-mentioned storage units may be stored by another storage unit.

[0041] <Variation 1> In the above embodiment, the deletion rate of past messages and the length of the summary are set uniformly for all users, but the present invention is not limited to this. The deletion rate and the length of the summary may be adaptively changed for each user.

[0042] For example, the message generating unit 212 may gradually increase the deletion rate according to the accumulated amount of dialogue history with the user. Specifically, the message generating unit 212 can increase the deletion rate when the number of past messages with the user exceeds a predetermined threshold. For example, the deletion rate can be set to increase by 10% when the number of past messages with the user exceeds 1000, and to increase by another 10% when the number exceeds 2000. In this way, the longer the dialogue with the user, the more actively the older the past messages are deleted, so that the storage amount can be adaptively controlled according to the user.

[0043] Furthermore, the message generating unit 212 may adaptively change the deletion rate and the length of the summary based on the characteristics of the user's responses. For example, if the user's responses are mostly short sentences, the message generating unit 212 can increase the deletion rate and shorten the length of the summary. This is because it can be determined that the user prefers concise responses. On the other hand, if the user's responses are mostly long sentences, the deletion rate and lengthen the length of the summary. This is because it can be determined that the user prefers detailed responses. In this way, by adapting to the characteristics of the user's responses, a dialogue that matches the user's preferences can be realized.

[0044] Furthermore, the message generating unit 212 may adaptively change the deletion rate and the length of the summary based on the attribute information of the user. For example, when the user is older, the message generating unit 212 can decrease the deletion rate and increase the length of the summary. This is because older users tend to have poorer memories. On the other hand, when the user is younger, the deletion rate can be increased and the length of the summary can be shortened. This is because younger users tend to have better memories. In this way, by taking the attribute information of the user into consideration, a dialogue that matches the characteristics of the user can be realized.

[0045] <Variation 2> In the above embodiment, the deletion and summarization of past messages are performed uniformly regardless of the contents of the messages, but the present invention is not limited to this. The method of summarization may be changed depending on the contents of the deleted messages.

[0046] For example, if the past message to be deleted is a response to a question from the user, the summarizing unit 213 can summarize the past message in detail without deleting it. Specifically, the summarizing unit 213 can determine the question from the user based on the presence or absence of an interrogative word (e.g., "what," "where," "who," "when," "why," etc.) in the past message and the presence or absence of a symbol at the end of the sentence (e.g., "?", etc.). Then, for the past message determined to be a response to the question from the user, a summary message with a length of 80% or more of the original past message can be generated without deleting it. In this way, since the response to the question from the user is considered to be of high importance in the dialogue, it is possible to prevent information loss by summarizing in detail.

[0047] When the past message to be deleted is small talk-like, the summarization unit 213 can actively delete the past message or only roughly summarize it. Specifically, the summarization unit 213 can determine the small talk-like content based on the presence or absence of a specific keyword (e.g., "weather," "mood," "small talk," etc.) in the past message. Then, for past messages determined to be small talk-like, the deletion rate can be set high or the length of the summary message can be reduced to about 20% of the original past message. In this way, since small talk-like content is considered to be less important in a dialogue, unnecessary references in the message generation unit 212 can be reduced by actively deleting the small talk-like content or roughly summarizing it.

[0048] Furthermore, the summarization unit 213 can extract important keywords from the past messages to be deleted, and generate a summary message so as to include the keywords. Specifically, the summarization unit 213 can analyze the past messages with a morphological analyzer, and extract nouns, proper nouns, unknown words, and the like as important keywords. Then, the summarization unit 213 can generate a summary message so as to include as many of the extracted keywords as possible. This allows important information contained in the past messages to be deleted to be reflected in the summary message.

[0049] <Variation 3>

[0050] The method of summarization may be changed based on the result of emotion estimation.

[0051] For example, the summarization unit 213 can change the length of the summary message based on the type of emotion estimated by the emotion estimation unit 214. Specifically, the summarization unit 213 can summarize in more detail a past message in which a positive emotion such as joy or surprise has been detected. For example, the summarization unit 213 can generate a summary message with a length that is 60% or more of the original past message in which a past message in which an emotion such as joy or surprise has been detected. In this way, since past messages in which the user has expressed positive emotions are considered to be of high importance in a dialogue, the user's emotions can be appropriately reflected by summarizing in detail.

[0052] The summarizing unit 213 can make do with a rough summary of a past message in which a negative emotion such as sadness or anger is detected. For example, the summarizing unit 213 can generate a summary message with a length of about 30% of the original past message in which a past message in which a sadness or anger is detected is detected. In this way, since it is considered preferable that a user does not remember in detail a past message in which a negative emotion is expressed in a dialogue, the influence of negative content can be minimized by making a rough summary.

[0053] The summarizing unit 213 can continuously change the length of the summary message based on the degree of emotion estimated by the emotion estimation unit 214. Specifically, the summarizing unit 213 can lengthen the summary message as the degree of emotion estimated from the past message is higher. For example, the summarizing unit 213 can be set to generate a summary message with a length of 80% or more of the original past message for a past message with a degree of joy of 0.8 or more and less than 0.8, and to generate a summary message with a length of 60% or more of the original past message for a past message with a degree of joy of 0.6 or more and less than 0.8. In this way, by continuously changing the length of the summary message according to the degree of emotion, the user's emotion can be reflected in more detail.

[0054] The summarizing unit 213 can change the style of the summary message based on the type of emotion estimated by the emotion estimation unit 214. Specifically, the summarizing unit 213 can generate a summary message in a more casual style for past messages in which positive emotions such as joy and surprise have been detected. For example, exclamatory expressions such as "Amazing!" and "Good job!" can be included. On the other hand, the summarizing unit 213 can generate a summary message in a more formal style for past messages in which negative emotions such as sadness and anger have been detected. For example, formal expressions such as "That's unfortunate" and "I'm sorry" can be used. In this way, by changing the style of the summary message depending on the type of emotion, a dialogue that is in line with the user's emotions can be realized.

[0055] <Disclosures> The present disclosure also includes the following configurations. [Item 1] a message storage unit that stores past messages that are messages transmitted to and received from a user in association with the user; a message generating unit that generates the transmission message by providing a prompt including the past message corresponding to the user who is a conversation partner and an instruction to generate a transmission message which is a message to be newly transmitted to the user to a large-scale language model; Equipped with the message generating unit deletes a portion of the past message corresponding to the conversation partner user from the prompt; An information processing system comprising: [Item 2] The information processing system according to item 1, a summary unit for generating a summary message summarizing the past messages to be deleted; the message generating unit includes the summary message in the prompt in place of the past message to be deleted; An information processing system comprising: [Item 3] Item 2. An information processing system according to the present invention, the summarizing unit creates a summary message that is shorter the older the past message is; An information processing system comprising: [Item 4] Item 2. An information processing system according to the present invention, the summarizing unit providing a prompt, the prompt including an instruction to summarize the past message and the past message, to the large-scale language model to generate the summary message; An information processing system comprising: [Item 5] The information processing system according to item 1, a feeling estimation unit that estimates a degree of feeling of the user based on the first message; the message generating unit does not delete the past messages having a degree of emotion equal to or greater than a predetermined value; An information processing system comprising: [Item 6] storing past messages, which are messages transmitted and received between the user and the user, in association with the user; generating a transmission message by providing a large-scale language model with a prompt including the past message corresponding to the user as a conversation partner and an instruction to generate a transmission message which is a new message to be transmitted to the user; The computer executes In the step of generating the outgoing message, the computer deletes from the prompt a portion of the past message corresponding to the other user; An information processing method comprising: [Explanation of symbols]

[0056] 1 User terminal 2 Management Server

Claims

1. a message storage unit that stores past messages that are messages transmitted to and received from a user in association with the user; a message generating unit that generates the transmission message by providing a prompt including the past message corresponding to the user who is a conversation partner and an instruction to generate a transmission message which is a message to be newly transmitted to the user to a large-scale language model; an emotion estimation unit that estimates an emotion level of the user based on the past messages; Equipped with the message generating unit deletes a part of the past messages corresponding to the dialogue partner user from the prompt, and does not delete the past messages having a degree of emotion equal to or greater than a predetermined value; An information processing system comprising:

2. 2. The information processing system according to claim 1, a summary unit for generating a summary message summarizing the past messages to be deleted; the message generating unit includes the summary message in the prompt in place of the past message to be deleted; An information processing system comprising:

3. 3. The information processing system according to claim 2, the summarizing unit providing a prompt including an instruction to summarize the past message and the past message to the large-scale language model to generate the summary message; An information processing system comprising:

4. storing past messages, which are messages transmitted to and received from a user, in association with the user; generating a transmission message by providing a large-scale language model with a prompt including the past message corresponding to the user as a conversation partner and an instruction to generate a transmission message which is a new message to be transmitted to the user; estimating an emotional level of the user based on the past messages; The computer executes In the step of generating the transmission message, the computer deletes a part of the past messages corresponding to the conversation partner user from the prompt, and does not delete the past messages having a degree of emotion equal to or greater than a predetermined value; An information processing method comprising:

Images