Instant messaging method, system and related device

Abstract

The embodiments of the present application provide an instant messaging method, system and related devices. The instant messaging method includes: in the process of instant messaging between a designated account and a target account, obtaining at least the emotional style information of the latest instant messaging message sent by the target account; obtaining a temporary context prompt word corresponding to the emotional style information; wherein the temporary context prompt word is a guiding text generated based on the emotional style information, used to express the content requirements for the instant messaging message; at least based on the latest instant messaging message sent by the target account and the prompt instruction constructed by the temporary context prompt word, calling a generative speech large model to generate an instant messaging message including voice data; wherein the content of the instant messaging message conforms to the semantics expressed by the temporary context prompt word. The instant messaging method, system and related devices can improve the emotional fit of the automatically generated instant messaging message to the user to a certain extent.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer network communication technology, and more particularly to an instant messaging method, system and related devices. Background Art

[0002] In the field of instant messaging, with the development of Internet technology, people are increasingly using instant messaging software for daily communication. In the field of professional services, such as medical consultation, education, customer service, etc., instant messaging has gradually become an important means of communication.

[0003] In order to improve communication efficiency, some automated message generation tools have been introduced in the prior art. For example, in the medical field, patients send health consultation questions to doctors' accounts through instant messaging software, and the system automatically generates reply messages from doctors' accounts, such as explanations of common diseases and preliminary suggestions. This automated answering method can reduce the time taken up by doctors and enable patients to get professional responses in a timely manner. However, in the actual communication process, the messages sent by patients usually carry specific emotional states, such as anxiety, tension, anger, etc. This emotional state has an important impact on the communication effect.

[0004] Existing automatic message generation tools do not have information about the emotional state expressed by instant messaging messages during instant messaging, making it difficult for the instant messaging messages replied during instant messaging to meet the needs of emotional care. Summary of the invention

[0005] In view of this, multiple embodiments of the present application are dedicated to providing an instant messaging method, system and related devices, which can improve the emotional fit of automatically generated instant messaging messages to users to a certain extent.

[0006] One embodiment of the present application provides an instant messaging method, comprising: during instant messaging between a designated account and a target account, obtaining at least the emotional style information of the latest instant messaging message sent by the target account; obtaining a temporary context prompt word corresponding to the emotional style information; wherein the temporary context prompt word is a guiding text generated based on the emotional style information, and is used to express content requirements for the instant messaging message; at least based on the latest instant messaging message sent by the target account and the prompt instruction constructed by the temporary context prompt word, calling a generative speech large model to generate an instant messaging message including voice data; wherein the content of the instant messaging message conforms to the semantics expressed by the temporary context prompt word.

[0007] One embodiment of the present application provides an instant messaging system, including: an emotional style acquisition module, used to obtain at least the emotional style information of the latest instant messaging message sent by the target account during the instant messaging process between the designated account and the target account; a context prompt word acquisition module, used to obtain temporary context prompt words corresponding to the emotional style information; wherein the temporary context prompt words are guiding texts generated based on the emotional style information, and are used to express content requirements for instant messaging messages; a message generation module, used to call a generative speech large model based on at least the latest instant messaging message sent by the target account and the prompt instructions constructed by the temporary context prompt words, and generate an instant messaging message including speech data; wherein the content of the instant messaging message conforms to the semantics expressed by the temporary context prompt words.

[0008] One embodiment of the present application provides a computer device, which includes a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the instant messaging method as described above.

[0009] One embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by a processor, the instant messaging method as described above can be implemented.

[0010] One embodiment of the present application provides a computer program product, which is used to implement the instant messaging method as described above.

[0011] In multiple implementations provided in the present application, by obtaining the emotional style information of the latest instant messaging message sent by the target account, a temporary context prompt word corresponding to the emotional style is generated, and a prompt instruction is constructed based on the prompt word to call a generative speech model to generate an instant messaging message including voice data, so that the generated voice content can accurately reflect the semantics expressed by the temporary context prompt word, so that the generated instant messaging message can be more in line with the emotional state of the user corresponding to the target account, thereby enhancing the user's interactive experience in communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of an instant messaging system provided for one embodiment of the present application.

[0013] Figure 2 An interactive schematic diagram of an instant messaging method provided for one embodiment of the present application.

[0014] Figure 3 A flowchart of an instant messaging method provided for one embodiment of the present application.

[0015] Figure 4 A schematic diagram of modules of an instant messaging system provided for one embodiment of the present application.

[0016] Figure 5 A schematic diagram of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0018] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0019] In related technologies, large language models have been widely used in various fields. Through training on a large amount of data, large language models have good language understanding and generation capabilities, and can be used for information exchange and content generation in different scenarios. Specifically, using the built prompt instructions to call the large language model can generate answers corresponding to user questions, which plays an important role in the fields of instant messaging, automatic question and answer, etc. For example, in the field of instant messaging, users can conduct various consultations through chat software, and the instant messaging system can use the large language model to generate automatic reply content to meet the user's communication needs.

[0020] However, in the actual instant messaging process, the messages sent by users often carry different emotional states, such as anxiety, tension, anger, etc. These emotional states will have a significant impact on the communication effect. Existing automated message generation tools can usually only generate text or voice replies based on user questions, but cannot identify the user's emotional state and adjust the tone and intonation of the generated content accordingly, making it difficult to provide an interactive experience with emotional care. For example, when user consultation involves complex issues, the automatic reply of the instant messaging system may make the communication process appear stiff and fail to meet the user's emotional needs in communication.

[0021] For example, in medical consultation scenarios, patients consult doctors about health issues through instant messaging software, and their messages often contain emotional information, such as anxiety or worry. When generating replies, existing instant messaging systems usually only explain the condition and ignore the patient's emotional state. As a result, patients may feel that the content of the reply is stiff and lacks comfort, which will affect their satisfaction with communication and their acceptance of advice.

[0022] After studying existing instant messaging technologies, researchers have discovered that automatically generated reply messages can significantly improve communication effectiveness if they can combine the emotional state of the user's instant messaging messages to generate voice content that is more in line with the emotion.

[0023] See also Figure 1 . In multiple implementations provided in the present application, the instant messaging system may include a client and a server. The client may be an electronic device with network access capability. Specifically, for example, the client may be a desktop computer, a laptop computer, a tablet computer, a smart phone, a digital assistant, a smart wearable device, a shopping guide terminal, a smart speaker, a microphone and other devices. Among them, smart wearable devices include but are not limited to smart bracelets, smart watches, smart glasses, smart helmets, smart necklaces, etc. Alternatively, the client may also be a software application that can run in the above-mentioned electronic devices, such as an instant messaging software user terminal, a medical consultation APP, an educational counseling tool, etc.

[0024] The server can be an electronic device with certain computing and processing capabilities, equipped with hardware resources such as network communication modules, processors and memories. The server can also be a distributed server, including multiple processors, memories, network communication modules, etc., which work together to achieve various functions. Alternatively, the server can also be a server cluster formed by several servers, with higher computing and data processing capabilities. With the development of science and technology, the server can also be implemented by new forms of technical means, such as a new type of "server" based on quantum computing, to meet the needs of large-scale data processing and generative speech large model calls in the process of instant communication.

[0025] See also Figure 1 and Figure 2 . One embodiment of the present application provides an example of an application scenario of an instant messaging system. The instant messaging system may include a client and a server to support communication between a specified account and a target account and the generation, sending and receiving of instant messaging messages. The client may be an electronic device with network access capabilities, such as a desktop computer, a laptop computer, a tablet computer, a smart phone, etc. The server may be an electronic device with certain computing and data processing capabilities, which can call a generative speech model to achieve complex sentiment analysis and speech synthesis.

[0026] In this scenario example, the designated account is a doctor's account, and the target account can be a patient's account. The patient may use the instant messaging function of the instant messaging system to send a text or voice message to the doctor to ask about some disease-related issues. The server can automatically generate an instant messaging message to reply to the patient and give feedback to the patient in the name of the doctor's account, thus reducing the doctor's time occupation.

[0027] For example, a patient may send an instant messaging message to a doctor's account through a patient client: "Doctor, I have been feeling chest tightness recently and I don't know why." After receiving the instant messaging message, the server may store it and analyze it to identify the emotional style information in the instant messaging message. In this embodiment, the emotional style information may be obtained as "anxiety" by calling a large language model to analyze the content of the patient's instant messaging message.

[0028] After the server identifies the target account's emotional style information, it can match temporary context prompt words corresponding to the emotional style information based on a pre-built context prompt word set in the server. For example, if the emotional style information is identified as "anxiety", the temporary context prompt words corresponding to the emotional style information can be "use a soothing tone and provide a simple explanation".

[0029] Furthermore, the server can match the language habit feature words related to the temporary situation prompt words in the language habit feature set of the specified account. For example, if a doctor often uses soothing language habit feature words such as "the situation is fine" and "you don't need to worry too much" when communicating with patients.

[0030] The server can construct a prompt instruction, including the instant messaging message sent by the patient, the temporary context prompt word, and the matched language habit feature word, and then call the generative speech model to generate the reply content. In this embodiment, the generative speech model includes a content generation module and a speech synthesis module. The content generation module generates the reply text content that conforms to the emotion and context based on the prompt instruction. For example, against For the patient's message, the content generation module may generate a reply like this: "Don't worry too much. Chest tightness may be caused by a variety of reasons. We can check step by step." The speech synthesis module will combine the reply text provided by the content generation module and the doctor's personalized voice characteristics to convert the reply text into voice data. The speech synthesis module will refer to the doctor's voice timbre, speaking speed, intonation and other characteristics, and adjust the voice data according to the tone requirements in the temporary situation prompt words, so that the generated voice sounds soothing and in line with the doctor's expression habits.

[0031] Finally, the instant messaging system sends the instant messaging message including the voice data to the patient's client in the name of the doctor's account, and the patient receives and plays the voice data on the patient's client. The reply heard by the patient will present the doctor's unique voice and tone, achieving the effect that the instant messaging system automatically communicates with the patient in the name of the doctor.

[0032] In multiple rounds of conversations between doctors and patients, the instant messaging system will re-analyze sentiment, match contextual prompts, select language habit feature words, and generate content based on each instant messaging message sent by the patient, so that each round of instant messaging messages generated can meet the patient's current emotional state and communication needs. The instant messaging system can also continuously collect patient feedback and conversation content, continuously optimize and improve the process of emotional style information analysis and contextual prompt word matching, making the communication effect more natural.

[0033] See also Figure 3 The embodiment of the present application provides an instant messaging method. The instant messaging method can be applied to an instant messaging system. The instant messaging method includes the following steps.

[0034] Step S110: During the instant communication between the designated account and the target account, at least the emotional style information of the latest instant communication message sent by the target account is obtained.

[0035] In this embodiment, the emotional style information is used to describe the emotional state of the user corresponding to the target account during the instant messaging process. For example, the emotional state represented by the emotional style information includes but is not limited to anxiety, tension, happiness, anger, etc. The instant messaging system can generate the emotional style information based only on the instant messaging messages of the target account, or it can use the instant messaging messages of the target account and the specified account to determine the emotional style information. Instant messaging messages are not limited to text messages, but can also be voice messages. For voice messages, the instant messaging system can first identify the text content from the voice data included in the voice message, and then generate the emotional style information.

[0036] In this embodiment, during the process of instant messaging between the designated account and the target account, the instant messaging system can identify the emotional style information corresponding to the instant messaging message in each round of conversation. The instant messaging message may refer to the information containing text, symbols, voice and other contents sent by the target account during the communication process. The instant messaging message may carry the current emotional state of the user corresponding to the target account. In order to accurately obtain the emotional style information, the instant messaging system can use a pre-trained natural language processing model, or call a large language model to analyze the instant messaging message content of the target account and identify the corresponding emotional style information.

[0037] For example, in the first round of dialogue, the target account sent a voice message: "Doctor, I always feel chest tightness recently and I am very scared." After receiving this voice message, the instant messaging system first converts it into text. Then, the instant messaging system can construct a prompt instruction including the text and submit it to the large language model to instruct the large language model to generate emotional style information. Specifically, the large language model analyzes this text according to the prompt instruction, and the generated emotional style analysis can be "anxiety" in combination with its context and vocabulary usage. In the subsequent second round of dialogue, the target account sent a new text message: "Do I need to do some examinations?" The instant messaging system obtains this new instant messaging message, constructs the prompt instruction again, calls the large language model to analyze the instant messaging message, and derives the emotional state as "doubtful" or "slightly nervous". In this way, the instant messaging message can regenerate emotional style information in each round of dialogue to more accurately guide the instant messaging message to generate an instant messaging message as a reply.

[0038] In some embodiments, the instant messaging system can also allow multiple rounds of conversations to share the same emotional style information. For example, in the first round of conversation, the target account sends a message: "I have severe insomnia recently and I can't fall asleep." The instant messaging system calls the large language model for analysis and generates the emotional style information "anxiety". In the next few rounds of conversation, the target account sends several messages in succession: "Is there something wrong with my body?", "Do I need to take sleeping pills?", "Will insomnia affect my health?" The instant messaging system generates the corresponding reply instant messaging messages based on the emotional style information generated first. Of course, in some embodiments, the instant messaging system can also regenerate the emotional style information every few rounds of conversation.

[0039] Step S120: obtaining a temporary context prompt word corresponding to the emotional style information; wherein the temporary context prompt word is a guiding text generated based on the emotional style information, and is used to express content requirements for instant messaging messages.

[0040] In this embodiment, the temporary context prompt words can be guiding texts generated based on the emotional style information of the target account, and are used to guide the generation of instant messaging messages for the specified account. The temporary context prompt words can be dynamically generated based on the current context and emotional state during the instant messaging process, and can be continuously updated in each round of the conversation according to the changes in the emotional style information. The temporary context prompt words can guide the instant messaging system to generate a reply to the target account that more accurately matches the emotional state of the target account. Specifically,

[0041] In this embodiment, temporary situation prompt words can be generated by the instant messaging system using pre-built rules, or by calling a large language model. Specifically, the instant messaging system can construct prompt instructions based on the currently recognized emotional style information, and then call the large language model to generate temporary situation prompt words. For example, if the emotional style information of the target account is "nervous", the instant messaging system may generate temporary situation prompt words, including "adopting a soothing tone" and "using a simple and gentle explanation". For example, in the first round of conversation, the target account sent a voice message: "Doctor, I always feel chest tightness and fear recently." After receiving the instant messaging message, the instant messaging system converts it into text and analyzes the emotional state as "anxiety". Further, the instant messaging system can generate temporary situation prompt words based on the emotional state "anxiety", such as "use a soothing tone, provide detailed explanations, and avoid using professional terms." The temporary situation prompt words can only be for this round of communication context, which belongs to temporarily generated guidance information. In the second round of conversation, the target account sent a new instant messaging message: "Do I need to do some checks?" The instant messaging system identified the new emotional state as "doubtful" or "slightly nervous", and then temporarily generated new temporary situational prompts, such as "use an explanatory tone, provide inspection suggestions, and keep information transparent." Because the emotional state has changed, the new temporary situational prompts will replace the temporary situational prompts of the previous round to guide the instant messaging messages generated in this round.

[0042] In some embodiments, the temporary context prompt words can remain unchanged in multiple rounds of conversations until the emotional state of the user corresponding to the target account changes. For example, in consecutive rounds of conversations, the user corresponding to the target account has been showing an "anxious" state, and the instant messaging system can use the temporary context prompt words identified in the first time until the instant messaging system identifies a new emotional state. Since the temporary context prompt words are temporary, the instant messaging system can flexibly respond to different communication situations, and temporarily generate the most appropriate guiding text according to the latest emotional state in each round of conversation, ensuring that the instant messaging message better meets the emotional needs of the user corresponding to the target account.

[0043] Step S130: Based at least on the latest instant messaging message sent by the target account and the prompt instruction constructed by the temporary context prompt word, the generative speech model is called to generate an instant messaging message including voice data; wherein the content of the instant messaging message conforms to the semantics expressed by the temporary context prompt word.

[0044] In this embodiment, after acquiring the emotional style information of the target account and generating the corresponding temporary context prompt words, the instant messaging system further constructs a prompt instruction based on the latest instant messaging message sent by the target account and the temporary context prompt words to call the generative speech model to generate an instant messaging message including speech data. The content of the generated instant messaging message conforms to the semantics expressed by the temporary context prompt words to meet the emotional needs of the target account and provide a more natural communication experience.

[0045] In this embodiment, the prompt instruction can be an instruction for guiding the generative speech model to output voice data, which contains relevant information about the target account's instant messaging messages and temporary context prompt words. Specifically, the instant messaging system can first obtain the latest instant messaging message of the target account, and then combine the instant messaging message with the corresponding temporary context prompt word to construct a prompt instruction that can instruct the generative speech model how to generate the reply content. For example, if the temporary context prompt word is "use a soothing tone and provide a detailed explanation", the instant messaging system will emphasize this requirement when constructing the prompt instruction, so that the generative speech model can reflect the corresponding tone and content direction when generating voice data.

[0046] For example, in the first round of conversation, the target account sent a message: "Doctor, I have been feeling chest tightness lately and I am very scared." The instant messaging system analyzed the emotional state as "anxiety" and generated a temporary situational prompt word "Use a soothing tone, provide detailed explanations, and avoid using professional terms." Then, the instant messaging system combined the instant messaging message with the temporary situational prompt word, constructed a prompt instruction, and submitted it to the generative speech big model. According to the prompt instruction, the generative speech big model generates the content of the instant messaging message: "Please don't worry too much. Chest tightness may be caused by many reasons, such as excessive stress and indigestion. We can do some further examinations to clarify the cause, and then take corresponding measures based on the examination results." Then, the generative speech big model outputs this content in the form of voice data with a soothing tone. The instant messaging system can use the voice data as the message content of the specified account's response to the target account's instant messaging message. In the subsequent second round of conversation, the target account sent a new instant messaging message: "Do I need to do some examinations?" The instant messaging system identifies the emotional state as "doubtful" or "slightly nervous" and generates new temporary context prompts, such as "Use an explanatory tone, provide inspection suggestions, and keep information transparent." The instant messaging system combines this round of instant messaging messages with the new temporary context prompts, builds new prompt instructions, and calls the generative speech model to generate voice data. When generating voice data, the generative speech model follows the guiding information in the temporary context prompts and generates the content of the instant messaging message as follows: "In order to better understand your situation, we recommend that you first do an electrocardiogram and some basic blood tests. These tests can help us rule out some common problems." Then, the generative speech model converts this content into voice data with an explanatory tone. The instant messaging system can use the voice data as the message content of the instant messaging message of the specified account in response to the target account.

[0047] In some embodiments, the instant messaging system can maintain the consistency of the prompt instructions used in multiple rounds of conversations until it recognizes that the emotional state of the target account has changed and generates a new temporary situational prompt word. In this way, the generative speech model can use the same temporary situational prompt word in multiple consecutive rounds of conversations. For example, in multiple rounds of conversations, the emotional state of the target account continues to be "worried", the instant messaging system will maintain the temporary situational prompt word initially generated, and continue to reflect instructions such as "use an encouraging tone" in subsequent prompt instructions to guide the generative speech model to generate coherent and considerate replies, such as: "Don't worry, we will check step by step to ensure that your health status receives the best attention. In many cases, these symptoms can be alleviated with some simple treatments."

[0048] In multiple implementations provided in the present application, by obtaining the emotional style information of the latest instant messaging message sent by the target account, a temporary context prompt word corresponding to the emotional style is generated, and a prompt instruction is constructed based on the prompt word to call a generative speech model to generate an instant messaging message including voice data, so that the generated voice content can accurately reflect the semantics expressed by the temporary context prompt word, so that the generated instant messaging message can be more in line with the emotional state of the user corresponding to the target account, thereby enhancing the user's interactive experience in communication.

[0049] In some implementations, the instant messaging system may at least obtain the message content text of the instant messaging message most recently sent by the target account; and construct a prompt instruction based on the message content text to instruct the large language model to generate the emotional style information.

[0050] In this embodiment, the instant messaging system can at least obtain the message content text of the latest instant messaging message sent by the target account. The message content text may refer to the text content in the instant messaging message used to express the user's intention and emotional state. In some cases, the message content text can be directly obtained from the text type instant messaging message sent by the target account. In some embodiments, when the instant messaging message is a voice message, the instant messaging system can first extract the text content from the voice message, and then generate the emotional style prompt words based on the text content. For example, the target account sends an instant messaging message: "Recently, the work pressure is particularly high, and I always feel heavy in my heart." After receiving the instant messaging message, the instant messaging system obtains the content text of the message, that is, "Recently, the work pressure is particularly high, and I always feel heavy in my heart." Subsequently, the instant messaging system can construct a prompt instruction, which contains the content text and is accompanied by a prompt for emotional state recognition, such as "Please analyze the emotional state in this text." Then, the instant messaging system submits the prompt instruction to the large language model to instruct the large language model to generate emotional style information. The large language model analyzes the content text according to the prompt instruction, and combines the vocabulary and context in the prompt instruction to generate emotional style information such as "frustrated" or "stressed". Or, for example, the target account sends an instant messaging message: "Recently, the work pressure is particularly high, and I always feel heavy." The instant messaging system obtains the content text of the message, that is, "Recently, the work pressure is particularly high, and I always feel heavy." At the same time, the designated account previously replied to the instant messaging message: "Don't worry too much, I think you have done a good job." The instant messaging system uses these two instant messaging messages as input to construct a prompt instruction, which contains the message content text of the target account and the message content text of the reply instant messaging message of the designated account, and is accompanied by prompt information for emotional state recognition, such as "Please analyze the emotional state in the conversation.", to generate emotional style information.

[0051] In this embodiment, the instant messaging system can accurately identify the emotional state of the target account. Furthermore, the instant messaging system can not only identify the emotions of the target account itself, but also take into account the overall context of the communication between the two parties, so that the generated emotional style information is more comprehensive, providing an important emotional basis for the subsequent instant messaging message generation.

[0052] In some embodiments, the instant messaging system can match the language habit feature word corresponding to the temporary context prompt word in the language habit feature set of the designated account; wherein the language habit feature set includes at least one language habit feature word; the language habit feature word is used to represent the language expression habit of the user represented by the designated account; the language habit feature words corresponding to different temporary context prompt words are at least partially different; accordingly, based on the latest instant messaging message sent by the target account, the guidance text information and the language habit feature word corresponding to the temporary context prompt word, the generative speech big model is called to generate an instant messaging message including voice data.

[0053] In this embodiment, the language habit feature set of the designated account can be obtained by the instant messaging system based on the analysis of the historical instant messaging messages of the designated account. The instant messaging system can pre-collect the instant messaging messages sent by the designated account during the instant messaging process, analyze the contents of these instant messaging messages, and extract the language expressions unique to the designated account. The language habit feature words in the language habit feature set can include commonly used modal particles, idiomatic phrases, pause methods, etc., reflecting the language expression characteristics of the designated account in different situations. The instant messaging system can better imitate the personalized language style of the designated account in the subsequent instant messaging message generation process, thereby achieving more natural communication.

[0054] When the target account sends an instant messaging message, the instant messaging system obtains the emotional style information of the instant messaging message and generates corresponding temporary context prompt words. Then, the instant messaging system matches the language habit feature words related to the current temporary context prompt words in the language habit feature set of the specified account. Since different temporary context prompt words may involve different communication situations and emotional states, the instant messaging system will select the most suitable language habit feature words according to the current situation. For example, in a soothing communication situation, the matched feature words may be "relax" or "don't worry", while in an explanatory communication situation, the matched feature words may be "usually" or "it is recommended that you try it first".

[0055] In this embodiment, after the instant messaging system matches the language habit feature words, it can construct a prompt instruction based on the latest instant messaging message of the target account, the temporary situation prompt words and the matched language habit feature words, and submit the prompt instruction to the generative speech big model to generate an instant messaging message including voice data. The prompt instruction is used to guide the generative speech big model on how to generate the reply content, so that the generated message not only conforms to the semantics expressed by the temporary situation prompt words, but also can reflect the language style characteristics of the specified account. For example, the specified account is a doctor, and the modal particles frequently used in its historical instant messaging messages include "don't worry" and "take your time". The instant messaging system pre-collects and analyzes the conversation records of the doctor when communicating with the patient, extracts these feature words, and constructs the language habit feature set of the specified account. When the target account sends an instant messaging message: "Doctor, I always feel very tired recently, what should I do?" The instant messaging system analyzes the emotional state as "worry" and generates a temporary situation prompt word "use a soothing tone and provide a detailed explanation". The instant messaging system then matches the feature words related to the soothing tone in the language habit feature set of the specified account, such as "don't worry" and "take your time". Subsequently, the instant messaging system combines the target account's message content text, temporary contextual prompt words, and matched language habit feature words to construct a prompt instruction and submit it to the generative speech model. Based on the prompt instruction, the generative speech model generates the content of the instant messaging message: "Don't worry, this fatigue may be caused by a variety of reasons. We can do it step by step, and do some simple checks to eliminate common problems." Then, the generative speech model outputs this content in the form of voice data with a soothing tone. The instant messaging system uses the generated voice data as the content of the instant messaging message of the designated account in response to the target account.

[0056] In some embodiments, the instant messaging system extracts language habit feature words of the designated account from historical instant messaging messages generated when the designated account communicates with different target accounts; and adds the language habit feature words to a language habit feature set corresponding to the designated account.

[0057] In this embodiment, the language habit feature words can be used to represent the unique language habits shown by the designated account during the instant communication process. The instant communication system can add these extracted language habit feature words to the language habit feature set of the designated account. For example, the user corresponding to the designated account is a doctor, who usually uses some soothing sentences when communicating with patients, such as "Don't worry too much", "We can check step by step", etc. The instant communication system can collect the historical instant communication messages between these doctors and patients, and extract the language habit feature words that reflect the doctor's language habits from the historical instant communication messages, such as "Don't worry too much" and "Check step by step", and add these language habit feature words to the doctor's language habit feature set. The instant communication system continuously extracts and accumulates the language habit feature words used by doctors in the process of instant communication with different patients. In this way, in the subsequent process of generating instant communication messages, the instant communication system can better imitate the doctor's language expression habits when generating reply messages for different patients, so that the generated instant communication messages not only meet the semantic requirements of temporary situation prompt words, but also can show the doctor's unique language expression style.

[0058] In some embodiments, the language habit feature words may include: modal particles corresponding to different emotional style information extracted from the historical instant communication messages of the designated account; or, oral idioms corresponding to the designated account extracted from the historical instant communication messages of the designated account; or, pause habit words indicating language pause habits identified from the historical instant communication messages of the designated account.

[0059] Interjections can be used to express the attitude of the user corresponding to the specified account during the instant communication process. The instant communication system can extract the corresponding interjections from the historical instant communication messages of the specified account according to different emotional style information. Interjections can convey emotions and intentions, and can also play a role in easing, emphasizing or guiding emotions in communication. For example, the user corresponding to the specified account is a doctor, and the user corresponding to the target account is a patient. During the communication between the doctor and the patient, if the patient's emotional style information is identified as "nervous", the doctor may use soothing interjections such as "don't worry" and "it's okay". The instant communication system can extract these interjections related to specific emotional styles by analyzing the doctor's historical instant communication messages, and add them to the doctor's language habit feature set, so that the doctor's emotional care can be reflected when the instant communication messages are generated later.

[0060] Verbal idioms can be idiomatic phrases that a specified account often uses in communication. Verbal idioms can focus more on the specific way of expressing content, rather than being limited to conveying emotions. For example, when communicating with patients, doctors may often use phrases such as "it's fine", "it's okay", "stay optimistic", "it's not a big problem", or "it's a common situation". These phrases reflect the doctor's communication style. By analyzing the words used by doctors in historical instant messaging messages, the instant messaging system can extract these personalized verbal idioms and include them in the doctor's language habit feature set. In this way, in the subsequent instant messaging messages generated, the instant messaging system can imitate the doctor's professional communication style and provide replies that are closer to the doctor's style.

[0061] Pause habit words can be used to indicate the words used by the user corresponding to the specified account when there is a pause in the communication process. For example, when a doctor is communicating with a patient, he or she may use phrases such as "You can think about it" or "Pay attention to this when you go back", followed by a short language pause. The instant messaging system can identify pause habit words by analyzing the doctor's historical instant messaging messages and add pause habit words to the language habit feature set, so that when generating voice data later, it can imitate the communication rhythm of the user corresponding to the specified account and increase the realism of the conversation.

[0062] In some embodiments, the generative speech model includes an integrated content generation module and a speech synthesis module; wherein the content generation module is used to generate a reply content text based on the received prompt instruction, and the reply content text conforms to the semantics expressed by the temporary situation prompt word; the speech synthesis module is used to combine the reply content text and the sound personalized features corresponding to the designated account to generate speech data corresponding to the reply content text; wherein the sound tone in the speech data conforms to the semantics expressed by the temporary situation prompt word.

[0063] In this embodiment, the generative speech model includes an integrated content generation module and a speech synthesis module, which work together to generate reply content text and convert the reply content text into speech data.

[0064] In this embodiment, the content generation module can be used to generate a reply content text that conforms to the current context according to the prompt instruction. After receiving the prompt instruction constructed by the instant messaging system, the content generation module will generate a reply content text for responding to the target account according to the instant messaging message of the target account, the temporary context prompt words, and the language habit feature words of the specified account contained in the prompt instruction. In a specific embodiment, the content generation module may include a semantic analysis component, a text generator, and a language habit adjuster. The semantic analysis component can be used to identify the key semantics of the instant messaging message of the target account, and determine the semantic direction of the reply content text in combination with the temporary context prompt words. By parsing the instant messaging message of the target account, the semantic analysis component can obtain the main needs and emotional state of the target account to generate a reply content text that is more in line with the communication context. The text generator can generate a reply content text based on the prompt instruction provided by the instant messaging system. The text generator can be a large language model that generates a text reply with certain emotions and tones by combining and referring to the temporary context prompt words. For example, if the temporary context prompt words are "use an explanatory tone and provide detailed instructions", the text generator will generate a reply text containing explanatory content. The language habit adjuster can adjust the reply text so that the reply content conforms to the language habits of the user corresponding to the specified account. Specifically, the language habit adjuster optimizes the vocabulary and expressions in the reply text according to the language habit feature set of the specified account. For example, when the specified account is a doctor, the language habit adjuster may add phrases such as "the situation is fine" or "we can check step by step" that conform to the doctor's habits to the text, so that the subsequent speech generation is more in line with the expression style of the specified account.

[0065] In this embodiment, the speech synthesis module is used to convert the reply content text generated by the content generation module into voice data. In a specific embodiment, the speech synthesis module may include: a personalized sound library, a speech synthesis engine, a tone regulator, and a pause controller. The personalized sound library may include voice features of the user corresponding to the specified account. Such as timbre information, speech speed information, intonation information, etc. These sound features are extracted from the historical voice data of the specified account by the instant messaging system, and are used to simulate the voice characteristics of the user corresponding to the specified account, so that the generated voice data sounds more like the real voice of the user corresponding to the specified account. The speech synthesis engine can be used to convert the reply content text into voice data. Specifically, the speech synthesis engine can realize speech synthesis based on deep learning speech synthesis technology, such as Tacotron, WaveNet, etc. In the process of converting the reply text content into voice data, the speech synthesis engine will combine the sound features in the personalized sound library to ensure that the voice data conforms to the style of the user corresponding to the specified account in terms of timbre and intonation. In order to meet the tone requirements in the temporary situation prompt words, the tone regulator in the speech synthesis module will adjust the generated voice data. The tone regulator dynamically adjusts the speed, pitch, volume and other parameters in the voice data according to the semantic information of the temporary situation prompt words, such as "use a soothing tone" or "adopt an explanatory tone", so that the voice data sounds more in line with the current communication situation when it is played. When generating speech, the pause controller of the speech synthesis module determines the pause position and duration in the speech data based on the language habit feature words provided in the content generation module. The pause controller can add appropriate pauses to the speech by identifying the pause habit words in the reply content text, imitating the expression habits of the corresponding user of the specified account in communication. For example, when the reply content contains a pause habit word such as "you can think about it", the pause controller will insert a short pause after the phrase, making the voice data more realistic and natural when it is played.

[0066] In this embodiment, after the instant messaging system obtains the instant messaging message of the target account and generates the corresponding temporary situation prompt word, the content generation module generates the reply content text according to the prompt instruction and submits it to the speech synthesis module. The speech synthesis module combines the personalized sound characteristics of the specified account and the tone requirements in the temporary situation prompt word, converts the text into voice data, and finally generates voice data that meets the language style of the specified account and the emotional needs of the target account, and the voice data can be used as the message content of the instant messaging message. For example, the target account sends an instant messaging message: "Doctor, I always feel stomach discomfort recently and I am a little worried." The instant messaging system analyzes the emotional state of the target account as "anxiety" and generates a temporary situation prompt word "Use a soothing tone and provide a simple explanation." The content generation module generates a reply content text: "You don't have to worry too much. There are many common causes for this stomach discomfort. We can do some checks first." Then, the speech synthesis module generates voice data according to the reply content text and the sound characteristics in the personalized sound library, and the tone regulator adjusts the tone to be soothing. The pause controller adds a short pause after "You don't have to worry too much" to make the voice more in line with the doctor's expression style. Finally, the generative speech model outputs voice data with the doctor's personalized voice features, and the instant messaging system can send the voice message as the message content of the instant messaging message to the target account.

[0067] In some implementations, the instant messaging system may match the emotion style information with a context prompt word set to obtain a temporary context prompt word corresponding to the emotion style information; wherein the context prompt word set includes at least one temporary context prompt word.

[0068] In this embodiment, the instant messaging system can match the emotional style information of the target account in the context prompt word set to obtain a temporary context prompt word corresponding to the emotional state. Specifically, the context prompt word set can be a preset database in which a variety of temporary context prompt words are stored. Each temporary context prompt word can be associated with a specific emotional state. The emotional style information can correspond to one or more predefined temporary context prompt words. Each temporary context prompt word contains features such as language style, tone, content direction, etc. for the emotional state, so as to guide the subsequent instant messaging messages generated. For example, the context prompt words that may be associated with the "anxiety" state are "use a soothing tone, provide simple explanations, and reduce complex terms."

[0069] In this embodiment, even after the communication system obtains the emotional style information, it can search for temporary situational prompt words related to the emotional style information in the situational prompt word set. For example, in the situational prompt word set, temporary situational prompt words that may be associated with the "anxiety" state, such as "use a soothing tone and provide a simple explanation".

[0070] See also Figure 4 . One embodiment of the present application also provides an instant messaging system. Even if the communication system may include: an emotional style acquisition module, which is used to obtain at least the emotional style information of the latest instant messaging message sent by the target account during the instant messaging process between the designated account and the target account; a context prompt word acquisition module, which is used to obtain temporary context prompt words corresponding to the emotional style information; wherein the temporary context prompt words are guiding texts generated based on the emotional style information, and are used to express the content requirements for the instant messaging message; a message generation module, which is used to call the generative speech model based on at least the latest instant messaging message sent by the target account and the prompt instructions constructed by the temporary context prompt words, and generate an instant messaging message including voice data; wherein the content of the instant messaging message conforms to the semantics expressed by the temporary context prompt words.

[0071] In this embodiment, the specific functions and effects achieved by the instant messaging system can be explained with reference to other embodiments of the present application and will not be repeated here.

[0072] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the processor implements the instant messaging method as described above. .

[0073] The embodiment of the present application also provides a computer program product including instructions, and when the computer program product is executed by a processor, the instant messaging method as described above is implemented.

[0074] See also Figure 5 The present description embodiment may provide a computer device, the computer device comprising: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, the instructions are executed by the one or more processors, so that the one or more processors implement the instant messaging method as described above.

[0075] In some embodiments, the computer device may include a processor, a non-volatile storage medium, an internal memory, a communication interface, a display device, and an input device connected by a system bus. The non-volatile storage medium may store an operating system and related computer programs.

[0076] The user information or user account information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, etc.) involved in multiple implementation methods of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0077] It should be understood that the specific examples in this article are only intended to help those skilled in the art to better understand the embodiments of the present application, rather than to limit the scope of the present invention.

[0078] It can be understood that in the various implementations of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of the present application.

[0079] It can be understood that the various embodiments described in this application can be implemented individually or in combination, and the embodiments of this application are not limited to this.

[0080] Unless otherwise stated, all technical and scientific terms used in the embodiments of the present application have the same meaning as those generally understood by those skilled in the art of the technical field of the present application. The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the scope of the present application. The term "and / or" used in the present application includes any and all combinations of one or more related listed items. The singular forms of "a kind of", "above" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0081] It can be understood that the processor of the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method implementation can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0082] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0083] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0085] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0086] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0087] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0088] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0089] The above is only a specific implementation of the present application, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An instant messaging method, characterized in that: include: During the instant communication between the designated account and the target account, at least the emotional style information of the latest instant communication message sent by the target account is obtained; Acquire a temporary context prompt word corresponding to the emotional style information; wherein the temporary context prompt word is a guiding text generated based on the emotional style information and is used to express content requirements for instant messaging messages; In the language habit feature set of the designated account, the language habit feature word corresponding to the temporary context prompt word is matched; wherein the language habit feature set includes at least one language habit feature word; the language habit feature word is used to represent the language expression habit of the user represented by the designated account; the language habit feature words corresponding to different temporary context prompt words are at least partially different; Based on the latest instant messaging message sent by the target account, the guiding text information and the language habit feature words corresponding to the temporary situation prompt word, the generative speech big model is called to generate an instant messaging message including speech data; wherein the content of the instant messaging message conforms to the semantics expressed by the temporary situation prompt word; the generative speech big model comprises an integrated content generation module and a speech synthesis module; wherein the content generation module is used to generate a reply content text based on the received prompt instruction, and the reply content text conforms to the semantics expressed by the temporary situation prompt word; the speech synthesis module is used to generate speech data corresponding to the reply content text in combination with the sound personalized features corresponding to the designated account; wherein the sound tone in the speech data conforms to the semantics expressed by the temporary situation prompt word; Extracting language habit feature words of the designated account from historical instant communication messages generated by instant communication between the designated account and different target accounts; the language habit feature words include: modal particles corresponding to different emotional style information extracted from historical instant communication messages of the designated account; or, oral idiomatic words corresponding to the designated account extracted from historical instant communication messages of the designated account; or, pause habit words indicating language pause habits identified from historical instant communication messages of the designated account; The language habit feature word is added to the language habit feature set corresponding to the designated account.

2. The method according to claim 1, characterized in that The step of obtaining at least the emotional style information of the latest instant messaging message sent by the target account includes: At least obtaining the message content text of the latest instant messaging message sent by the target account; A prompt instruction constructed based on the message content text instructs the large language model to generate the emotional style information.

3. The method according to claim 1, characterized in that The step of obtaining a temporary context prompt word corresponding to the emotional style information comprises: The emotional style information is matched with a context prompt word set to obtain a temporary context prompt word corresponding to the emotional style information; wherein the context prompt word set includes at least one temporary context prompt word.

4. An instant messaging system, characterized in that: include: The emotional style acquisition module is used to acquire at least the emotional style information of the latest instant communication message sent by the target account during the instant communication between the designated account and the target account; A context prompt word acquisition module, used to acquire temporary context prompt words corresponding to the emotional style information; wherein the temporary context prompt words are guiding texts generated based on the emotional style information, and are used to express content requirements for instant messaging messages; A message generation module, used to match and obtain the language habit feature word corresponding to the temporary context prompt word in the language habit feature set of the designated account; wherein the language habit feature set includes at least one language habit feature word; the language habit feature word is used to represent the language expression habit of the user represented by the designated account; the language habit feature words corresponding to different temporary context prompt words are at least partially different; based on the latest instant communication message sent by the target account, the guiding text information and the language habit feature word corresponding to the temporary context prompt word, the generative speech big model is called to generate an instant communication message including voice data; wherein the content of the instant communication message conforms to the semantics expressed by the temporary context prompt word; the generative speech big model includes an integrated content generation module and a speech synthesis module; wherein the content generation module is used to generate a reply content text based on the received prompt instruction, and the reply content text conforms to the semantics expressed by the temporary context prompt word; the speech synthesis module is used to generate voice data corresponding to the reply content text in combination with the voice personalized features corresponding to the designated account; wherein the voice tone in the voice data conforms to the semantics expressed by the temporary context prompt word; Among them, the instant messaging system is also used to extract language habit feature words of the designated account from historical instant messaging messages generated when the designated account communicates with different target accounts; the language habit feature words include: modal particles corresponding to different emotional style information extracted from the historical instant messaging messages of the designated account; or, oral idioms corresponding to the designated account extracted from the historical instant messaging messages of the designated account; or, pause habit words indicating language pause habits are identified from the historical instant messaging messages of the designated account; and the language habit feature words are added to the language habit feature set corresponding to the designated account.

5. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the instant messaging method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by the processor, it can implement the instant messaging method according to any one of claims 1 to 3.

7. A computer program product, characterized in that The computer program product is used to implement the instant messaging method as claimed in any one of claims 1 to 3.

Images