Audio data generation method, instant communication method and related device

Abstract

The embodiments of the present application provide a method for generating audio data, an instant communication method, and related devices. The method comprises: storing a received text stream into a text buffer word by word in the order of reception; wherein the text stream comprises a plurality of characters; after the number of characters in the text buffer exceeds a specified word count threshold, and when a specified character is received, the characters in the text buffer are used as a sentence, and the audio data of the sentence is generated as a whole. The embodiments of the present application can improve the listening experience of the audio to a certain extent.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of instant messaging technology, and in particular to a method for generating audio data, an instant messaging method, and related devices. Background Art

[0002] In real life, people may need to convert text data into audio data, or convert audio data into text data in their daily life and work.

[0003] Generally, when converting text data into audio data, the full text of the text data is provided to an audio data conversion tool so that the audio data conversion tool can generate audio data corresponding to the text data in a better manner.

[0004] In some cases, the text data itself is in the form of a text stream that is gradually generated. In this case, it is difficult for existing audio data conversion tools to provide corresponding audio data.

[0005] Therefore, there is a problem in the prior art that the audio data generated for the text stream provides a poor listening experience for the user. Summary of the invention

[0006] In view of this, multiple embodiments of the present application are dedicated to providing an audio data generation method, an instant messaging method and related devices, which can enhance the user's listening experience of the audio corresponding to the text stream to a certain extent.

[0007] One embodiment of the present application provides a method for generating audio data, comprising: storing a received text stream into a text buffer word by word in the order of reception; wherein the text stream includes a plurality of characters; after the number of characters in the text buffer exceeds a specified word count threshold and specified characters are received, treating the characters in the text buffer as a sentence and generating audio data of the sentence as a whole.

[0008] One embodiment of the present application provides an instant messaging method, comprising: receiving question information sent by a consulting account in a target instant messaging message conversation item; displaying text included in a text stream generated for the question information according to a large language model; and playing audio data corresponding to a sentence in the text stream when audio data corresponding to a sentence in the text stream is received; wherein the audio data is generated by the aforementioned audio data generation method.

[0009] One embodiment of the present application provides an audio data generation device, including: a text stream receiving module, used to store the received text stream into a text buffer word by word in the order of reception; wherein the text stream includes multiple characters; a generation module, used to treat the characters in the text buffer as a sentence and generate audio data of the sentence as a whole when the number of characters in the text buffer exceeds a specified word count threshold and specified characters are received.

[0010] One embodiment of the present application provides an instant messaging device, comprising: an information receiving module, used to receive question information sent by a consulting account in a target instant messaging message conversation item; a text display module, used to display text included in a text stream generated for the question information according to a large language model; a playback module, used to play audio data corresponding to a sentence in the text stream when receiving audio data; wherein the audio data is generated according to the aforementioned audio data generation method.

[0011] One embodiment of the present application provides a computer device, which includes a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the method for generating audio data described in the above embodiment, or to implement the instant communication method described in the above embodiment.

[0012] One embodiment of the present application provides a computer-readable storage medium, in which at least one computer program is stored. When the at least one computer program is executed by a processor, it can implement the method for generating audio data described in the above embodiment, or implement the instant communication method described in the above embodiment.

[0013] The multiple implementation methods provided by the present application store the text in the text stream into a text buffer, divide it into appropriate sentences, and then generate audio data corresponding to the sentences as a whole. In this way, the generated audio data can better adapt the tone and speed of the sentences, so that the final played audio has a better listening experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A schematic diagram of the text-to-audio data processing process provided for one embodiment of the present application.

[0015] Figure 2 A flowchart of a method for generating audio data provided in accordance with one embodiment of the present application.

[0016] Figure 3 A schematic diagram of the modules of a system for generating audio data provided in accordance with one embodiment of the present application.

[0017] Figure 4 A schematic diagram of modules of an instant messaging device provided for one embodiment of the present application.

[0018] Figure 5 A schematic diagram of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0020] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0021] In the related art, large language models have been widely promoted and used. After training the large language model with a large number of training samples, the large language model can have a good reserve of professional knowledge and a deeper understanding of world knowledge. Furthermore, the large language model can be used to answer user questions. Specifically, the large language model can be called by constructing a prompt instruction to instruct the large language model to output the answer to the corresponding user question. The large language model usually outputs the answer text in the form of a text stream. That is, the large language model will output the text one by one, forming a text stream in the order of the text.

[0022] For example, in a specific application scenario, the large language model can be applied to the field of instant messaging technology. That is, in instant messaging technology, the intelligent question-and-answer robot account can communicate with the user and answer the questions raised by the user. Specifically, the large language model can be used to generate answers to the questions raised by the user.

[0023] In some cases, it may not be convenient for users to understand information by observing the text in the electronic terminal, such as when the user is older and has some visual impairment. In this case, the user is more inclined to understand the information by listening to audio.

[0024] After improving the instant messaging software, the R&D personnel have realized that the instant messaging client can play audio corresponding to the text displayed. Specifically, when the large language model outputs the text stream, each time a text is received, the text-to-audio tool is called to obtain the audio data of the corresponding text, and the audio data is sent to the instant messaging client for playback, thus meeting the user's usage needs.

[0025] However, the researchers further discovered that generating corresponding audio data for each word and sending it to the instant messaging client for playback did not provide a good user experience. Specifically, each word was independently generated audio data, which made the audio played by the client sound stiff and difficult to have emotional changes in tone, which gave the user a poor listening experience.

[0026] Therefore, it is necessary to provide a method for generating audio data, an instant communication method and related devices, which can enhance the listening experience of audio to users.

[0027] One embodiment of the present application provides a method for generating audio data. The method for generating audio data can be applied to an audio data generation system. The audio data generation system may include a client and a server. The client may be an electronic device with network access capability. Specifically, for example, the client may be a desktop computer, a tablet computer, a laptop computer, a smart phone, a digital assistant, a smart wearable device, a shopping guide terminal, a television, a smart speaker, a microphone, etc. Among them, smart wearable devices include but are not limited to smart bracelets, smart watches, smart glasses, smart helmets, smart necklaces, etc. Alternatively, the client may also be software that can be run in an electronic device. The server may be an electronic device with certain computing and processing capabilities. It may have a network communication module, a processor, and a memory, etc. The server may also be a distributed server, which may be a system with multiple processors, memories, network communication modules, etc. operating in collaboration. Alternatively, the server may also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server may also be a new technical means that can realize the corresponding functions of the implementation method of the specification. For example, it may be a new form of "server" based on quantum computing.

[0028] In some implementations, the audio data generation system may include multiple functional modules, and the functional modules may communicate with each other and share data.

[0029] One embodiment of the present application provides an instant messaging method. The instant messaging method can be applied to an instant messaging system. The instant messaging system may include a client and a server. The client may be an electronic device with network access capability. Specifically, for example, the client may be a desktop computer, a tablet computer, a laptop computer, a smart phone, a digital assistant, a smart wearable device, a shopping guide terminal, a television, a smart speaker, a microphone, etc. Among them, smart wearable devices include but are not limited to smart bracelets, smart watches, smart glasses, smart helmets, smart necklaces, etc. Alternatively, the client may also be software that can be run in an electronic device. The server may be an electronic device with certain computing and processing capabilities. It may have a network communication module, a processor, and a memory, etc. The server may also be a distributed server, which may be a system with multiple processors, memories, network communication modules, etc. operating in collaboration. Alternatively, the server may also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server may also be a new technical means that can realize the corresponding functions of the implementation method of the specification. For example, it may be a new form of "server" based on quantum computing.

[0030] In some implementations, the instant messaging system may include multiple functional modules, and the functional modules may communicate with each other and share data.

[0031] See also Figure 1 and Figure 2 The embodiment of the present application provides a method for generating audio data. The method for generating audio data can be used in a system for generating audio data, and of course can also be used only in a server of the system for generating audio data. The following description will only focus on the system for generating audio data. The method for generating audio data can include the following steps.

[0032] Step S110: storing the received text stream into the text buffer word by word in the order of receiving; wherein the text stream includes a plurality of characters.

[0033] In some cases, the text stream can be generated by calling a human intelligence module such as a large language model. And according to the content of the prompt instruction used when calling the large language model, the multiple words included in the text stream can be used as answers to specified questions, or introductions to specified subject content.

[0034] In this embodiment, the text stream may be a stream of text formed by a plurality of texts in sequence. The plurality of texts in the text stream are generated in a certain order. For example, a large language model outputs texts one by one according to probability. The plurality of texts may be formed into sentences, paragraphs or articles in the order in which they are output.

[0035] In this embodiment, the text buffer can be a buffer module used to temporarily store text. The text buffer can be implemented using a software data structure, such as a first-in-first-out queue. Of course, in some embodiments, the text buffer can also be implemented using a hardware cache memory.

[0036] In this embodiment, the content of the text stream can be received word by word, and accordingly, the words can be stored in the text buffer word by word. For example, the content of the text stream output by a large language model is "Taking a slow walk after a meal is beneficial to physical health". The audio data generation system can successively receive the words "饭 (meal)", "后 (after)",... "健 (health)" and "康 (well-being)", and the audio data generation system can store the words in the text buffer in the order in which the words are received.

[0037] Step S120: After the number of words in the text buffer exceeds a specified word count threshold and a specified word is received, the words in the text buffer are used as a sentence, and the audio data of the sentence is generated as a whole.

[0038] In some cases, there are certain response requirements for the generation time of the audio data corresponding to the text stream. For example, in the field of instant messaging, it is usually required to be as fast as possible. However, due to the characteristic that the text stream is received word by word, the subsequent words in the text stream are unknown. Considering the time response requirements, it is difficult to divide the sentences after analyzing the text formed by all the words as a whole. In the related art, although the response speed is improved by generating audio data word by word, the listening experience of the user is also poor.

[0039] In this embodiment, the specified word count threshold can be the minimum number of words used to divide a sentence. After receiving the words in the text stream word by word, the audio data generation system can also count the words in the process of putting the words into the text buffer, and after determining that the number of words is greater than the specified word count threshold, it can count whether the received words are specified words word by word. If the received words are specified words, the words in the text buffer can be combined into sentences. If the received words are not specified words, the received words will continue to be stored in the text buffer. In this way, the number of words divided into sentences is controlled by the specified word count threshold. It can avoid that the sentences are too short, resulting in incomplete semantic expression, or the sentences are too long and do not conform to normal reading habits. Of course, the specified word count threshold can be used to control the number of words in the sentences divided by the words included in the text stream, but due to the unknown characteristics of the subsequent words in the text stream, the last sentence of the text stream may not be constrained by the specified word count threshold, that is, the number of words in the last sentence corresponding to the text stream may be less than the specified word count threshold. In some embodiments, the specified word count threshold can be 13 words, 14 words, 15 words, 16 words, 17 words... 20 words, 21 words, etc., which will not be repeated. In some embodiments, the specified word count threshold when dividing the first sentence in the text stream is less than the specified word count threshold when dividing subsequent sentences. For example, the specified word count threshold when dividing the first sentence is 15 words, and the specified word count threshold when dividing subsequent sentences is 18 words. Since the audio data generation system converts the entire sentence into audio data, if the length of the first sentence is long, the time for the user to wait for the audio to be played will be very long, considering the time for generating the text stream, obtaining the entire sentence, and converting the entire sentence into audio data. By reducing the specified word count threshold of the first sentence, the waiting time of the user can be reduced to a certain extent, and the user experience can be improved.

[0040] In this embodiment, the specified text may be text used as the end of a sentence. In this embodiment, text may include characters and symbols that express semantics in a language. For example, taking Chinese as an example, text may include each Chinese character and punctuation marks. The specified text may be used to divide sentences so that the specified text usually serves as the end of a sentence. For example, the specified text may be a punctuation mark that expresses the end of a sentence, such as a period ".", a semicolon ";", an exclamation mark "!", a question mark "?" or a comma ",". In this embodiment, after the number of characters in the text buffer exceeds a specified word count threshold, and when the received text is considered to be the specified text, the characters in the text buffer are combined into sentences, so that through the specified text, it is determined that the characters in the text buffer can be used as a sentence that expresses appropriate semantics. This makes the division of sentences more accurate.

[0041] For example, the text stream includes the text "Cold accompanied by cough is a common symptom, usually caused by viral infections such as influenza or the common cold". The audio data generation system receives the above text stream verbatim, "感", "冒"..."通", "常". After receiving the character "常" and storing it in the text buffer, when the number of characters in the text buffer reaches the specified character count threshold of 15 characters, at this time, the audio data generation system continues to receive the characters "由", "病", "毒", "感", "染", "引", "起", ",". When the audio data generation system receives ",", it can form the text in the text buffer into the sentence "Cold accompanied by cough is a common symptom, usually caused by viral infections", and generate the audio data of the whole sentence as a whole.

[0042] In this embodiment, the entire sentence can be provided to the audio conversion module by calling the audio conversion module at one time, so as to convert the entire sentence into audio data. In this embodiment, by providing the entire sentence to the audio conversion, the audio conversion module can analyze the emotional tendency of the entire sentence through the relatively complete semantic content carried by the sentence itself, so that the generated audio data can combine the content of the sentence, making the audio more emotional and closer to the voice of a real person. In this embodiment, the audio conversion module can be a machine learning model that realizes the audio conversion function after being trained based on a large number of training samples. Of course, the audio conversion module can also be an audio conversion service provided by a third party. By calling the service interface of the audio conversion service, the entire sentence is provided to the audio conversion service, and the audio data fed back by the audio conversion service is received. In some embodiments, the audio data generated by the audio conversion module can also imitate the voice of a specified person and the speaking tone of a specified person. For example, an automatic reply function can be set for a doctor's consultation account to answer the questions consulted by patients. During the process of generating voice data, the voice data can be made to imitate the voice and speaking tone of the doctor, so that patients can have a better experience.

[0043] In some embodiments, the audio data generation system can obtain the question information of the consultation account in the target instant messaging message session item; construct a prompt instruction based on the question information, and the prompt instruction is used to instruct the large language model to generate a text stream corresponding to the question information; receive the text stream output by the large language model after calling the large language model using the prompt instruction.

[0044] In some cases, the instant messaging technology can be used as the entry of the audio data generation system. In this way, the audio data generation system is combined with the instant messaging technology to improve the user experience during instant messaging.

[0045] In this embodiment, accounts can establish instant messaging message session items, and collect account information and communication content for instant messaging through the instant messaging message session items to achieve instant messaging between accounts. The instant messaging message session item can also be used as a data set to record the instant messaging data generated by the participating accounts of the instant messaging. The instant messaging data may include but is not limited to text, pictures, audio data or video data, etc. Furthermore, the instant messaging message session item also indicates the account for instant messaging. That is, accounts belonging to the same instant messaging message session item can communicate in text, voice or video based on the instant messaging message session. The target instant messaging message session item may be the reply content of the question information of the consulting account in the instant messaging message session item, which needs to be converted into audio data and then provided to the consulting account.

[0046] In this embodiment, after the consulting user logs in to the instant messaging client, the account of the consulting user can be the consulting account. The consulting user can use the identity of the consulting account to operate the instant messaging client to send an instant messaging message. The content of the instant messaging message can be text information, picture information, audio information or video information, etc. In some cases, the consulting user can send an instant messaging message that requires the recipient to give an answer through the consulting account, and the instant messaging message can carry question information. For example, the instant messaging message sent by the consulting account is "How to recover health after catching a cold?", and this sentence is the question information. In some cases, the consulting user may send an instant messaging message through audio, and voice recognition can be performed on the corresponding audio message to obtain text question information.

[0047] In this embodiment, the audio data generation system can construct a prompt instruction after receiving the question information, so as to call the large language model through the prompt instruction. Specifically, the audio data generation system can be integrated with a prompt instruction template, and after receiving the question information, the question information is added to the prompt instruction module to form a prompt instruction. For example, the prompt instruction is "You are a doctor, please give a professional answer to the question "How to recover after catching a cold?"."

[0048] In this embodiment, after receiving the prompt instruction, the large language model will generate an answer according to the requirements of the prompt instruction and send the answer to the audio data generation system. In some embodiments, the large language model usually adopts a technical solution of word-by-word output, so that the word-by-word sent to the audio data generation system forms a text stream. Of course, the large language model can also output multiple words and then send the multiple words to the audio data generation system, and the audio data will still be received in the form of a text stream.

[0049] In this embodiment, the large language model may refer to an artificial intelligence model with a large number of parameters and a good learning ability of world knowledge. The large language model may be a generative model and can understand and generate natural language.

[0050] In some embodiments, after the number of characters in the text buffer exceeds a specified word count threshold and specified characters are received, the audio data generation system can assemble the characters buffered in the text buffer into a sentence in the order in which they are stored in the text buffer; and add the audio data generation task corresponding to the sentence to an audio task queue; wherein the audio task queue includes tasks that require the generation of audio data.

[0051] In some cases, the words in the text stream can only accurately express the original semantics if they are combined in order. Therefore, in the process of generating audio data, it is necessary to control the order of the audio data to avoid disorder of the order of the audio data, which makes it difficult for users to understand the meaning.

[0052] In this embodiment, after the audio data generation system divides the sentences, it can generate audio data generation tasks corresponding to the sentences. In this way, each sentence can correspond to an audio data generation task to ensure that audio data is generated for each sentence. Furthermore, by establishing an audio task queue, the first-in-first-out feature of the "queue" is utilized, so that the audio data generation tasks that enter the audio task queue first will be executed first, and the corresponding audio data can be generated according to the order of the sentences in the text stream. In this way, the audio data can be accurately generated according to the order of the sentences in the text stream.

[0053] In some implementations, after generating the audio data generation task for the corresponding sentence, the audio data generation system clears the text in the text buffer and continues to store the received text stream into the text buffer word by word in the order in which it is received.

[0054] In some cases, the audio data generation task requires a relatively long execution time to generate the audio data corresponding to the sentence. If the audio data generation system waits for the feedback result of the audio conversion module while calling the audio conversion module to generate audio data, the reception of the text stream will be interrupted or the text of the text stream will be lost.

[0055] In this embodiment, after the audio data generation system generates the audio data generation task for the corresponding sentence, it clears the text buffer, and can use the text buffer to receive new text, and can generate the audio data generation task based on the text in the text buffer as a sentence to achieve loop processing. Furthermore, the audio data generation system constructs the audio data generation task, adds the audio data generation task to the audio queue, and clears the text buffer and then re-stores the received text, thereby achieving the generation of audio data and the division of sentences, which are executed in parallel, and the overall efficiency is improved, while also taking into account the timeliness of receiving the text stream and the sequential accuracy of the generated audio data.

[0056] In some implementations, the audio data generation system may send the text included in the text stream to a target instant messaging client; and after the audio data corresponding to the sentence is generated, the audio data is sent to the target instant messaging client.

[0057] In some cases, during instant messaging, users often expect to receive a reply message as soon as possible.

[0058] In this embodiment, after receiving the text included in the text stream, the audio data generation system can send the received text to the target instant messaging client. Specifically, the audio data generation system can send each text to the target instant messaging client word by word, or after the audio data generation system divides the sentences, the sentences can be sent to the target instant messaging client. The target instant messaging client can be the instant messaging client used by the consulting user, or the instant messaging client logged into the consulting account. In this way, the target instant messaging client can get feedback as soon as possible after the user sends the question information.

[0059] In this embodiment, after the audio data generation system provides the text to the target instant messaging client, the audio data corresponding to the sentence can be generated and then sent to the target instant messaging client, so that the target instant messaging client can play the audio according to the audio data. In this way, the user can listen to the audio content when it is difficult to read the text, which brings convenience to the user. Furthermore, the entire sentence of the audio data is generated, so that the audio content is more coherent and closer to the expression of a real person, which improves the user experience.

[0060] In some implementations, the audio data generation system may execute the audio data processing task in the audio task queue to obtain the audio data corresponding to the sentence; and store the audio data in a designated audio database. Further, the audio data generation system may write the access identifier corresponding to the audio data in the designated audio database into the extended field of the instant messaging message to which the text in the text stream belongs; and upon receiving an access request for the access identifier from the target instant messaging client, send the audio data to the target instant messaging client.

[0061] In some cases, after the audio data generation system generates audio data, if the generated audio data is sent directly to the instant messaging client, if the sending fails, the audio data may be lost, and the audio data needs to be regenerated. The audio data of multiple sentences need to be provided to the target instant messaging client in the exact order, which will cause more problems.

[0062] In this embodiment, after the audio data generation system performs the audio data processing task, the audio data corresponding to the sentence will be obtained. The audio data generation system will store the audio data in a specified audio database, so that the audio data corresponding to the sentence will not be easily lost. Further, the audio data stored in the specified audio database may correspond to an access identifier, and the audio data can be obtained through the access identifier. Specifically, after the audio data is stored in the specified audio database, the specified audio database can provide an access identifier for the audio data in the specified audio database. The text included in the text stream is sent to the target instant messaging client via an instant messaging message so that the target instant messaging client can display the content of the instant messaging message, and the audio data generation system can write the access identifier of the audio data into an extended field of the instant messaging message. The extended field does not need to be displayed in the interface of the target instant messaging client.

[0063] In this embodiment, after obtaining the access identifier, the target instant messaging client can send an access request for the access identifier to the specified audio database to obtain the corresponding audio data, and can play the audio after obtaining the audio data. Further, in some embodiments, the target instant messaging client will receive the text in the text stream word by word, and can add a text to the instant messaging session interface after each text is received, so as to achieve the display effect of dynamically adding text to the instant messaging message, reduce the overall waiting time of the user, increase the interest of reading, and enhance the user experience. Furthermore, in the process of the target instant messaging client dynamically adding text to display the instant messaging message, when the extended field of the instant messaging message is written with the access identifier, the target instant messaging client can obtain the corresponding audio data according to the access identifier, and can play the audio data corresponding to the sentence formed in the text stream in the process of displaying the instant messaging message in a dynamically added text manner. In this way, the user can understand the content of the instant messaging message visually and auditorily, providing the user with a better user experience.

[0064] In this embodiment, the designated audio database can store data objects. The audio data can be stored in the designated audio database as data objects.

[0065] The embodiment of the present application also provides an instant communication method. The instant communication method may include: receiving question information sent by a consulting account; displaying the text included in the text stream according to the text stream generated by the large language model for the question information; and playing the audio data corresponding to the sentence in the text stream when receiving the audio data; wherein the audio data is generated according to any of the aforementioned audio data generation methods.

[0066] In this embodiment, the instant messaging method can be applied to an instant messaging client. The instant messaging client can log in to a robot account. For example, the robot account can be a question-and-answer robot account. The consulting account can be an account that establishes an instant messaging message session item with the robot account. The consulting account can perform instant messaging with the robot account based on the instant messaging message session item.

[0067] In this embodiment, the question information carries the content of the question raised by the consulting account. The consulting account may send the question information through a text message or a voice message. The instant messaging client may receive a text stream generated by the large language model corresponding to the question information, and display the text included in the text stream. Specifically, the instant messaging client may receive the text included in the text stream word by word, and increase and display it in the interface word by word. Of course, the instant messaging client may also receive multiple texts, such as a sentence, before displaying the corresponding sentence in the interface.

[0068] In this embodiment, when the instant messaging client receives audio data corresponding to a sentence in a text stream, the corresponding audio data can be played, thus bringing convenience to the user.

[0069] In some embodiments, when the consulting account sends question information in audio form, the instant messaging client needs to convert the sentences contained in the corresponding text stream into audio data. It is understandable that the data form of the question information sent by the corresponding consulting account can be used to feedback the instant messaging message in the corresponding data form. For example, it may be inconvenient for the user of the consulting account to send question information in text form, and at the same time, the user may have certain difficulties in reading, such as certain visual impairment of the user. The corresponding consulting account sends question information in audio form, and when replying to the instant messaging message of the consulting account, the form of audio playback is also provided, so that the user does not need to view the text, which brings convenience to the user.

[0070] See also Figure 3 The embodiment of the present application also provides an audio data generation system, comprising: a text stream receiving module, used to store the received text stream into a text buffer word by word in the order of reception; wherein the text stream includes a plurality of characters; a generation module, used to treat the characters in the text buffer as a sentence and generate audio data of the sentence as a whole when the number of characters in the text buffer exceeds a specified word count threshold and specified characters are received.

[0071] Regarding the specific functions and effects achieved by the audio data generation system, reference can be made to other embodiments of the present application for comparative explanation, which will not be repeated here. Each module in the audio data generation system can be implemented in whole or in part by software, hardware, and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0072] See also Figure 4 . The embodiment of the present application also provides an instant messaging device, comprising: an information receiving module, used to receive question information sent by a consulting account in a target instant messaging message conversation item; a text display module, used to display the text included in the text stream according to the text stream generated by the large language model for the question information; a playback module, used to play the audio data corresponding to the sentence in the text stream when receiving the audio data; wherein the audio data is generated according to the audio data generation method described in any of the aforementioned embodiments.

[0073] The specific functions and effects realized by the instant messaging device can be explained with reference to other embodiments of the present application, and will not be described here. Each module in the instant messaging device can be implemented in whole or in part by software, hardware, and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0074] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor implements any one of the methods for generating audio data, or implements any one of the instant messaging methods.

[0075] The embodiment of the present application also provides a computer program product including instructions, which, when executed by a processor, enables a computer to implement any of the above-mentioned methods for generating audio data, or to implement any of the above-mentioned instant messaging methods.

[0076] See also Figure 5 . The present description embodiment may provide a computer device, the computer device comprising: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, the instructions are executed by the one or more processors to enable the one or more processors to implement the audio data generation method in any of the above embodiments, or to implement the instant communication method in any of the above embodiments.

[0077] In some embodiments, the computer device may include a processor, a non-volatile storage medium, an internal memory, a communication interface, a display device, and an input device connected by a system bus. The non-volatile storage medium may store an operating system and related computer programs.

[0078] The user information or user account information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, etc.) involved in multiple implementation methods of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0079] It should be understood that the specific examples in this article are only intended to help those skilled in the art to better understand the embodiments of the present application, rather than to limit the scope of the present invention.

[0080] It can be understood that in the various implementations of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of the present application.

[0081] It can be understood that the various embodiments described in this application can be implemented individually or in combination, and the embodiments of this application are not limited to this.

[0082] Unless otherwise stated, all technical and scientific terms used in the embodiments of the present application have the same meaning as those generally understood by those skilled in the art of the technical field of the present application. The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the scope of the present application. The term "and / or" used in the present application includes any and all combinations of one or more related listed items. The singular forms of "a kind of", "above" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0083] It can be understood that the processor of the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method implementation can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0084] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0085] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0086] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0087] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0089] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0090] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0091] The above is only a specific implementation of the present application, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for generating audio data, characterized in that: include: The received text stream is stored word by word in a text buffer in a receiving order; wherein the text stream includes a plurality of characters; After the number of characters in the character buffer exceeds a specified character count threshold and a specified character is received, the characters buffered in the character buffer are combined into a sentence in the order in which they are stored in the character buffer; Adding the audio data generation task corresponding to the sentence to an audio task queue; wherein the audio task queue includes tasks that require the generation of audio data; Sending the text included in the text stream to a target instant messaging client; Execute the audio data processing task in the audio task queue to obtain the audio data corresponding to the sentence; Storing the audio data in a specified audio database; Writing the access identifier corresponding to the audio data in the designated audio database into the extension field of the instant messaging message to which the text in the text stream belongs; When receiving the access request for the access identifier sent by the target instant messaging client, the audio data is sent to the target instant messaging client.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining question information of the consulting account in the target instant messaging message conversation item; Constructing a prompt instruction based on the question information, the prompt instruction is used to instruct the large language model to generate a text stream of answer information corresponding to the question information; After the large language model is called using the prompt instruction, a text stream output by the large language model is received.

3. The method according to claim 1, characterized in that The method further comprises: Clear the text in the text buffer; Continue to store the received text stream into the text buffer word by word in the order in which it is received.

4. An instant messaging method, characterized in that: include: Receive question information from consulting accounts; According to the text flow generated by the large language model for the question information, display the text included in the text flow; When audio data corresponding to a sentence in the text stream is received, the audio data is played; wherein the audio data is generated according to any one of the audio data generation methods according to claims 1 to 3.

5. A system for generating audio data, characterized in that: include: A text stream receiving module, used for storing the received text stream into the text buffer word by word in the receiving order; wherein the text stream includes a plurality of characters; A generation module is used to, after the number of characters in the character buffer exceeds a specified character count threshold and specified characters are received, combine the characters buffered in the character buffer into a sentence in the order in which they are stored in the character buffer; add an audio data generation task corresponding to the sentence to an audio task queue; wherein the audio task queue includes tasks that require the generation of audio data; send the characters included in the character stream to a target instant messaging client; execute the audio data processing tasks in the audio task queue to obtain the audio data corresponding to the sentence; store the audio data in a specified audio database; write an access identifier corresponding to the audio data in the specified audio database into an extended field of the instant messaging message to which the characters in the character stream belong; and upon receiving an access request for the access identifier issued by the target instant messaging client, send the audio data to the target instant messaging client.

6. An instant messaging device, characterized in that: include: An information receiving module, used for receiving question information sent by a consulting account in a target instant messaging message session item; A text display module, used for displaying the text included in the text flow according to the text flow generated by the large language model for the question information; A playing module is used to play the audio data corresponding to the sentence in the text stream when the audio data is received; wherein the audio data is generated according to any one of the audio data generation methods according to claims 1 to 3.

7. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the method for generating audio data as described in any one of claims 1 to 3, or to implement the instant communication method as described in claim 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by the processor, it can implement the method for generating audio data as described in any one of claims 1 to 3, or implement the instant communication method as described in claim 4.

9. A computer program product, characterized in that The computer program product is used to implement the method for generating audio data as described in any one of claims 1 to 3, or to implement the instant communication method as described in claim 4.

Images