Speech synthesis method and apparatus, storage medium, and electronic device

Abstract

The application discloses a speech synthesis method and device, a storage medium and an electronic device, and relates to the technical field of smart homes. The method comprises the following steps: converting speech data in a terminal interaction request from a terminal device into text data, and obtaining text response data indicated by an analysis result of the text data; obtaining audio pre-synthesis data obtained by performing a TTS pre-synthesis operation on the text response data; in the case that the audio pre-synthesis data is found, sending response information generated based on the audio pre-synthesis data to the terminal device; in the case that the audio pre-synthesis data is not found, obtaining audio synthesis data obtained by performing a TTS synthesis operation on the text response data, and storing the audio synthesis data in an audio cache library. The above technical solution solves the technical problem that speech synthesis cannot be quickly and accurately performed.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and more specifically, to a speech synthesis method, apparatus, storage medium, and electronic device. Background Technology

[0002] Currently, with the rapid development of artificial intelligence technology, especially the advancement of large-scale AI models, human-computer voice interaction is increasingly widely used in the smart home field. This interaction method provides users with a more natural and convenient operating experience through voice and other media. However, voice interaction requires real-time audio synthesis, resulting in a slow response speed. To improve response speed, related technologies employ a strategy of caching synthesized voice data. The basic idea of ​​this strategy is to pre-generate and store some commonly used voice data so that a quick response can be provided when a user requests it. However, the pre-stored data in this method is insufficient to cover the user's actual interaction needs, resulting in a low hit rate and significantly reducing the effectiveness of caching, thus hindering fast and accurate voice synthesis.

[0003] Therefore, there are technical problems in related technologies that prevent the rapid and accurate synthesis of speech.

[0004] No effective solution has yet been proposed to address the technical problem of speech synthesis that cannot be performed quickly and accurately in related technologies. Summary of the Invention

[0005] This application provides a speech synthesis method, apparatus, storage medium, and electronic device to at least solve the technical problem in the related art that speech synthesis cannot be performed quickly and accurately.

[0006] According to one embodiment of this application, a speech synthesis method is provided, comprising: converting a speech packet in a terminal interaction request from a terminal device into text data, and obtaining text response data indicated by the parsing result of the text data; obtaining audio pre-synthesized data obtained after performing a text-to-speech (TTS) pre-synthesis operation on the text response data; if the audio pre-synthesized data is found, sending response information generated based on the audio pre-synthesized data to the terminal device; if the audio pre-synthesized data is not found, obtaining audio synthesized data obtained after performing a TTS operation on the text response data, and storing the audio synthesized data in the audio cache library.

[0007] In one exemplary embodiment, the method further includes: identifying hot-spot interaction requests from interaction log records, wherein the hot-spot interaction requests include at least a first interaction request and a second interaction request, the request frequency of the first interaction request is higher than that of a first interaction request with a preset frequency, and the second audio pre-synthesized data corresponding to the second interaction request is not found in the audio cache library; performing aggregation processing on the first audio pre-synthesized data corresponding to the first interaction request to obtain a first aggregation result, and performing aggregation processing on the second audio pre-synthesized data to obtain a second aggregation result; and pre-caching the first aggregation result and the second aggregation result.

[0008] In an exemplary embodiment, before obtaining the audio pre-synthesized data obtained after performing a TTS pre-synthesizing operation on the text response data, the method further includes: calling a preset audio synthesis service to perform a TTS pre-synthesizing operation on the text response data and obtaining the audio pre-synthesized data generated after performing the TTS pre-synthesizing operation; and pre-caching the audio pre-synthesized data in the audio cache library.

[0009] In one exemplary embodiment, obtaining the text response data indicated by the parsing result of the text data includes: parsing the text data using a natural language processing model to obtain interaction intent keywords and determining the intent type corresponding to the interaction intent keywords; obtaining intent data corresponding to the intent type from a database; and determining the text response data based on the intent data and the interaction intent keywords.

[0010] In an exemplary embodiment, after determining the text response data based on the intent data and the interaction intent keywords, the method further includes: performing a data correction operation on the text response data to obtain corrected data; and determining the corrected data as the text response data, wherein the data correction operation includes at least one of the following: a scene replacement operation, or a script update operation.

[0011] In one exemplary embodiment, the data correction operation includes a script update operation. Performing a data correction operation on the text response data to obtain corrected data includes: performing a script update operation on the text response data to obtain corrected data by: modifying the text description script of the text response data using a default text description script to obtain corrected data; or, for a pre-launch database, obtaining the pre-launch script of the pre-launch database, and using the pre-launch script to modify the text description script of the text response data to obtain corrected data.

[0012] In one exemplary embodiment, the data correction operation includes a scene replacement operation. Performing a data correction operation on the text response data to obtain corrected data includes: performing a scene replacement operation on the text response data in the following manner: when it is determined that the intent scene corresponding to the intent type has been updated, synchronously updating the intent data; determining the text response data based on the updated intent data and the interaction intent keyword; and performing a data correction operation on the text response data to obtain corrected data.

[0013] According to another aspect of the embodiments of this application, a speech synthesis apparatus is also provided, comprising: an acquisition module, configured to convert speech packets in a terminal interaction request from a terminal device into text data, and acquire text response data indicated by the parsing result of the text data; a search module, configured to acquire audio pre-synthesized data obtained after performing a text-to-speech (TTS) pre-synthesis operation on the text response data; a sending module, configured to send response information generated based on the audio pre-synthesized data to the terminal device if the audio pre-synthesized data is found; and a storage module, configured to acquire audio synthesized data obtained after performing a TTS operation on the text response data if the audio pre-synthesized data is not found, and store the audio synthesized data in the audio cache library.

[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described speech synthesis method when it is run.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-described speech synthesis method through the computer program.

[0016] In this embodiment, the process involves converting voice packets from a terminal interaction request from a terminal device into text data and obtaining text response data indicated by the parsing result of the text data; obtaining audio pre-synthesized data after performing a text-to-speech (TTS) pre-synthesis operation on the text response data; if the audio pre-synthesized data is found, sending response information generated based on the audio pre-synthesized data to the terminal device; if the audio pre-synthesized data is not found, obtaining audio synthesized data after performing a TTS operation on the text response data and storing the audio synthesized data in the audio cache library. By dynamically updating the audio synthesized data in the cache, the system can quickly respond to user requests during voice interaction, improving cache hit rate and timely playback of corresponding audio. This dynamic cache update strategy shortens interaction time by trading space for time, providing a more efficient interactive experience and improving the real-time performance of human-computer interaction. It solves the technical problem of not being able to perform speech synthesis quickly and accurately, achieving not only fast and accurate speech synthesis but also improving the processing efficiency of user interaction requests. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the hardware environment of a speech synthesis method according to an embodiment of this application;

[0020] Figure 2 This is a flowchart of a speech synthesis method according to an embodiment of this application;

[0021] Figure 3 This is a schematic diagram illustrating the implementation process of the speech synthesis method according to an embodiment of this application;

[0022] Figure 4 This is a structural block diagram of a speech synthesis device according to an embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] According to one aspect of the embodiments of this application, a speech synthesis method is provided. This speech synthesis method is widely used in whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligencehouse ecosystems. Optionally, in this embodiment, the above-mentioned speech synthesis method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0026] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0027] This embodiment provides a speech synthesis method applied to the aforementioned terminal device. Figure 2 This is a flowchart of a speech synthesis method according to an embodiment of this application, which includes the following steps:

[0028] Step S202: Convert the voice packet in the terminal interaction request from the terminal device into text data, and obtain the text response data indicated by the parsing result of the text data;

[0029] Among these, the parsing of text data can be understood as, for example, as performing intent parsing on the text data.

[0030] Intent parsing is performed using NLP (Natural Language Processing) models. Intent parsing is a crucial task in the field of natural language processing, aiming to identify and understand the intent behind user-input natural language text. Simply put, it's about enabling computers to understand what action a user wants to perform or what question they want to ask.

[0031] Step S204: Obtain audio pre-synthesized data after performing a text-to-speech (TTS) pre-synthesis operation on the text response data;

[0032] The aforementioned TTS pre-synthesis operation can be understood as TTS pre-synthesis, for example.

[0033] In this application, TTS (Text-to-Speech) is a technology that converts text information into audible speech, which can improve the performance of speech synthesis applications and the user experience. TTS technology is widely used in various scenarios, such as voice assistants, e-book readers, and navigation systems. In the context of real-time TTS caching, TTS represents real-time generated speech data.

[0034] Caching is a storage mechanism used to temporarily save data for quick access. Therefore, TTS real-time caching refers to storing real-time generated speech data for rapid playback or processing when needed. This caching mechanism can improve the response speed and efficiency of speech synthesis.

[0035] TTS pre-synthesis refers to converting text into speech files and saving them before the actual speech output is needed. This way, when the user needs to hear the speech, the system can directly play the pre-synthesized audio without real-time text-to-speech conversion. Pre-synthesis reduces real-time processing latency and improves response speed, especially under poor network conditions or limited computing resources.

[0036] TTS pre-caching refers to storing pre-synthesized speech files locally or on a server for quick access when needed. Pre-caching can reduce network latency and improve the smoothness of voice playback. For example, in a voice assistant application, developers might pre-synthesize some frequently used phrases or commands and cache these audio files on the user's device. When the user speaks these phrases, the application can directly play the cached audio without waiting for network requests or real-time synthesis.

[0037] By combining TTS pre-synthesis and TTS pre-caching, and by preparing and storing voice data in advance, the need for real-time processing is reduced, thereby improving response speed and user experience. This can significantly enhance the performance and user experience of TTS applications. For example, in an online reading application, developers might pre-synthesize chapters that the user is likely to read and play the audio directly while the user is reading, instead of synthesizing in real time. This reduces waiting time and improves the continuity of reading.

[0038] Step S206: If the audio pre-synthesized data is found, send the response information generated based on the audio pre-synthesized data to the terminal device;

[0039] Optionally, the audio pre-synthesized data can be retrieved from an audio cache library.

[0040] Step S208: If the audio pre-synthesized data is not found, obtain the audio synthesized data obtained after performing TTS operation on the text response data, and store the audio synthesized data in the audio cache library.

[0041] In this step, the aforementioned TTS synthesis operation may involve requesting a self-developed TTS algorithm or a third-party TTS service to synthesize TTS audio. Optionally, in this step, during voice interaction, the corresponding text can be obtained through the interfaces of data providers such as weather, news, knowledge bases, and music, and then the TTS speech synthesis service (i.e., a self-developed TTS algorithm or a third-party TTS speech synthesis service) can be used to synthesize the audio, and the synthesized audio can be played.

[0042] Through the above steps, the voice packets in the terminal interaction request from the terminal device are converted into text data, and the text response data indicated by the parsing result of the text data is obtained; audio pre-synthesized data is obtained after performing a text-to-speech (TTS) pre-synthesis operation on the text response data; if the audio pre-synthesized data is found, the response information generated based on the audio pre-synthesized data is sent to the terminal device; if the audio pre-synthesized data is not found, audio synthesized data is obtained after performing a TTS operation on the text response data, and the audio synthesized data is stored in the audio cache library. By dynamically updating the audio synthesized data in the cache, the cached audio synthesized data can be used to quickly respond to user requests when the user performs voice interaction. This not only improves the cache hit rate but also allows for timely playback of the corresponding audio to the user. This dynamic cache update strategy shortens the interaction time by trading space for time, providing a more efficient interactive experience, improving the real-time performance of human-computer interaction, and solving the technical problem of not being able to perform speech synthesis quickly and accurately. It not only achieves fast and accurate speech synthesis but also improves the processing efficiency of user interaction requests.

[0043] In an exemplary embodiment, before obtaining the audio pre-synthesized data obtained after performing a TTS pre-synthesizing operation on the text response data, a preset audio synthesis service may be invoked to perform a TTS pre-synthesizing operation on the text response data, and the audio pre-synthesized data generated after the TTS pre-synthesizing operation is obtained; the audio pre-synthesized data is then pre-cached in the audio cache library.

[0044] In this embodiment, the text response data can be pre-cached during the user's voice interaction. Since the terminal and other business scenarios may change the text response data, the pre-cached text response data is not the final text of the actual synthesized TTS. For example, the matching degree between the pre-cached text response data and the final text is between 60% and 70%.

[0045] Optionally, in one embodiment, during user interaction, if the pre-synthesized audio data is not found in the audio cache library, the cache is not hit, and the pre-synthesized audio data is immediately cached, for example, with a cache validity period of 1 hour. If the pre-synthesized audio data is found in the audio cache library, the cache is hit, and the cache validity period is reset, for example, if the original 1-day validity period now has 1 hour left before it expires, the validity period is reset to 1 day.

[0046] In an exemplary embodiment, the process of obtaining the text response data indicated by the parsing result of the text data may include: parsing the text data using a natural language processing model to obtain interaction intent keywords and determining the intent type corresponding to the interaction intent keywords; obtaining intent data corresponding to the intent type from the database; and determining the text response data based on the intent data and the interaction intent keywords.

[0047] In an exemplary embodiment, after determining the text response data based on the intent data and the interaction intent keywords, the text response data is further subjected to a data correction operation to obtain corrected data; the corrected data is then determined as the text response data, wherein the data correction operation includes at least one of the following: a scene replacement operation, or a script update operation.

[0048] In an exemplary embodiment, the data correction operation includes a script update operation. The technical solution for performing a data correction operation on the text response data to obtain corrected data specifically includes: performing a data correction operation on the text response data to obtain corrected data, specifically by: performing a script update operation on the text response data in the following ways to obtain corrected data: modifying the text description script of the text response data using a default text description script to obtain corrected data; or, for a pre-launch database, obtaining the pre-launch script of the pre-launch database, and using the pre-launch script to modify the text description script of the text response data to obtain corrected data.

[0049] In an exemplary embodiment, the data correction operation includes a scenario replacement operation. The process of performing a data correction operation on the text response data to obtain corrected data includes: performing a data correction operation on the text response data to obtain corrected data. Specifically, this is achieved by performing a scenario replacement operation on the text response data as follows: when it is determined that the intent scenario corresponding to the intent type has been updated, the intent data is updated synchronously; the text response data is determined based on the updated intent data and the interaction intent keyword; and a data correction operation is performed on the text response data to obtain corrected data. In this embodiment, by performing data correction on the text response data, real-time data updates can be achieved, thereby improving data accuracy.

[0050] In one exemplary embodiment, a technical solution is also proposed, specifically including: identifying hot-spot interaction requests from interaction log records, wherein the hot-spot interaction requests include at least a first interaction request and a second interaction request, the request frequency of the first interaction request is higher than that of a first interaction request with a preset frequency, and the second audio pre-synthesized data corresponding to the second interaction request is not found in the audio cache library; performing aggregation processing on the first audio pre-synthesized data corresponding to the first interaction request to obtain a first aggregation result, and performing aggregation processing on the second audio pre-synthesized data to obtain a second aggregation result; and pre-caching the first aggregation result and the second aggregation result.

[0051] Optionally, in one embodiment, the cache validity period, cache quantity, and cache size of the audio pre-synthesized data can be dynamically adjusted based on attributes such as timbre, volume, and speech rate of the audio pre-synthesized data, so as to manage and maintain the cached content.

[0052] In addition, this application can perform personalized and fine-grained differentiation and management of cached data based on user interaction requests, which can effectively improve the cache hit rate and reduce the storage pressure of speech synthesis and caching.

[0053] To better understand the process of the above speech synthesis method, the implementation flow of the above speech synthesis method will be described below in conjunction with optional embodiments, but this is not intended to limit the technical solution of the embodiments of this application.

[0054] This embodiment provides a speech synthesis method. Figure 3 This is a schematic diagram illustrating the implementation process of the speech synthesis method according to an embodiment of this application, such as... Figure 3 As shown, after the terminal picks up the audio, the ASR in the cloud recognizes the audio to obtain the text, performs NLP intent parsing on the text, and obtains the content text (i.e. the above text response data) based on the parsed intent. The final content text that is corrected and changed is then used for TTS audio synthesis.

[0055] Among them, the accurate acquisition of content text reflects the real-time nature of the data (it cannot be old data), and whether the TTS audio synthesis is correct and whether the synthesis takes a long time will affect the accuracy of subsequent cache hits. This application improves the interactive experience by accurately acquiring content text and improving the speed of TTS audio synthesis.

[0056] like Figure 3 As shown, this application includes real-time TTS caching, pre-caching of TTS, caching of TTS hot data, caching of fixed scripts, and advance caching of pre-launched scripts. The validity period of these types of caches can be configured and managed.

[0057] Real-time TTS caching means that if the current TTS synthesis fails to hit the cache during user interaction, it will be cached immediately after the TTS speech synthesis. Alternatively, if the current TTS synthesis hits the cache, the cache validity period corresponding to this text will be reset.

[0058] TTS pre-caching means that during user voice interaction, after speech recognition and intent recognition are parsed, the obtained text content (without modification or alteration) is immediately pre-synthesized and cached (at this time, the content text result matches the actual synthesized TTS text with a matching degree of 60% to 70%).

[0059] Hot data caching can include data that misses the cache and hot data from requests. It represents hot data (including synthesized text, corresponding intent domain, timbre, speech rate, volume, pitch, etc.) statistically collected from user interaction data according to time dimensions such as hourly, daily, and weekly. At different time dimensions, hot data is pre-cached periodically based on characteristics such as text domain and timbre (which can include the top N data that misses the cache and the top N data of TTS hot requests).

[0060] Fixed script (corresponding to default text description script) caching refers to caching fixed scripts (with long usage periods and not easily changed) defined according to various demand scenarios after audio synthesis. The caching period can be set from six months to one year, and dynamic online / offline operation is supported.

[0061] Pre-launch script caching refers to the pre-synthesis and caching of audio for pre-launch scripts set for pre-launch features. For example, when launching a knowledge base, the text in the knowledge base is synthesized using TTS (Text-to-Speech) and cached in advance.

[0062] Optionally, in one embodiment, the implementation process for TTS real-time caching and TTS pre-caching is as follows:

[0063] Step 1: The user initiates a normal voice interaction through smart home devices (speakers, refrigerators, air conditioners, water heaters, and kitchen appliances) or device software, and sends the audio data packet to the AI ​​cloud service.

[0064] Step 2: The AI ​​cloud service performs audio decoding and recognition, and parses the text data corresponding to the speech.

[0065] Step 3: The AI ​​cloud service performs NLP intent parsing on the text data to extract the text intent and the corresponding intent domain (corresponding intent type) and other data.

[0066] Step 4: Use data such as text intent and intent domain to obtain response text from the database (equivalent to the text response data mentioned above), such as weather, music, news, etc. data from third-party services, and then proceed to Step 5 (asynchronous) and Step 6 (synchronous);

[0067] Step 5: Asynchronously trigger the TTS pre-synthesis of the response text and perform TTS pre-caching.

[0068] Step 6: Simultaneously trigger the correction of the response text (the correction logic will be different for each terminal device) and the replacement of the scenario dialogue (involving dialogue platform and other related configurations) to obtain the final content response text;

[0069] Step 7: Request the TTS synthesis service to perform TTS audio synthesis on the final response text data.

[0070] If the audio is cached, retrieve the audio directly from the cache and return it.

[0071] If the audio is not cached, request a self-developed TTS algorithm or a third-party TTS service to synthesize the TTS audio, while caching the audio data synthesized for this TTS request in real time.

[0072] Optionally, in one embodiment, the process for caching TTS hotspot data, caching fixed scripts, and pre-launch caching is as follows:

[0073] Step 1: The TTS log statistics service performs data statistical analysis to obtain hot data. Hot data refers to TTS request data statistically analyzed by hour, day and week, including the Top N of cache misses and the Top N of high-frequency TTS requests. Steps 2 and 3 are triggered in parallel.

[0074] Step 2, as follows Figure 3 As shown, after the TTS caching operation service obtains log statistical analysis data for each time dimension, it performs aggregation processing of the Top N high-frequency TTS requests, and processes them into Top N data usable by TTS service according to synthesized text, timbre, speech rate, volume, etc., triggering the high-frequency TTS request pre-caching strategy.

[0075] Step 3: After obtaining the log statistics and analysis data for each time dimension, the TTS cache operation service performs aggregation processing on the Top N TTS requests that missed the cache, and processes them into Top N data (synthesized text, timbre, speech rate, volume, etc.) that can be used by the TTS service, triggering the TTS cache miss pre-caching strategy.

[0076] Step 4: After the pre-caching strategies in Steps 2 and 3 are triggered, the TTS service will be requested to perform the TTS strategy caching process. At this time, TTS audio synthesis and caching will be performed according to the request, so that subsequent actual TTS synthesis requests can obtain it. The caching period depends on the configuration of each time dimension in the TTS caching operation service.

[0077] Step 5, the process for caching fixed dialogue scripts and pre-launched dialogue scripts is basically similar, except that the source of the scripts is different. Fixed dialogue scripts are basically fixed interactive response scripts within the scope of smart home, while pre-launched dialogue scripts are scenario and feature scripts that are rolled over and iterated according to project needs. Specifically, it can be manually configured through the TTS caching strategy operation platform.

[0078] Step 6: Based on Step 5, after configuring the TTS pre-caching of synchronously triggered dialogue, the product synchronously requests the TTS service and proceeds to Step 7.

[0079] Step 7: After receiving the request, the TTS service performs TTS audio synthesis. If the synthesized audio can be cached, it is directly retrieved from the cache and returned. If it cannot be cached, a self-developed TTS algorithm or a third-party TTS service is used for TTS audio synthesis. At the same time, the audio data synthesized for this TTS request is cached in real time. The caching period can be manually configured according to the product in Step 5.

[0080] Based on the above embodiments, on the one hand, dynamic strategies for real-time caching and pre-caching are proposed for different fields and usage scenarios (fixed scripts, dynamic pre-released scripts), such as weather, knowledge base, news, and music. For example, weather is cached for only 3 hours, knowledge base for 1 week, and news for 1 day. This can cope with scenarios with a sudden increase in TTS requests, improve the cache hit rate when requesting the same text again, and improve the real-time TTS cache hit rate. On the other hand, TTS caching strategies for hot data, fixed scripts, and pre-released scripts can cope with scenarios of general TTS requests and long-tail TTS requests, improving the cache hit rate of periodic TTS requests.

[0081] In particular, the hot TTS caching strategy for TTS cache misses and high-frequency TTS requests can replenish cached data in a timely manner, improving the TTS cache hit rate. Therefore, this application can combine and coordinate the above caching strategies according to different needs and scenarios, improving the overall real-time performance and efficiency of TTS interaction during voice interaction, reducing the time and cost of TTS synthesis, and enhancing the voice interaction experience.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0083] Figure 4 This is a structural block diagram of a speech synthesis device according to an embodiment of this application; as shown below. Figure 4 As shown, it includes:

[0084] The acquisition module 42 is used to convert the voice packet in the terminal interaction request from the terminal device into text data, and acquire the text response data indicated by the parsing result of the text data;

[0085] The lookup module 44 is used to obtain the audio pre-synthesized data obtained after performing a text-to-speech (TTS) pre-synthesis operation on the text response data;

[0086] The sending module 46 is used to send response information generated based on the audio pre-synthesis data to the terminal device when the audio pre-synthesis data is found;

[0087] Storage module 48 is used to obtain audio synthesis data obtained after performing TTS operation on the text response data when the audio pre-synthesis data is not found, and to store the audio synthesis data in the audio cache library.

[0088] The above-described device converts voice packets from terminal interaction requests from terminal devices into text data and obtains text response data indicated by the parsing results of the text data. It then obtains pre-synthesized audio data obtained after performing a text-to-speech (TTS) pre-synthesis operation on the text response data. If the pre-synthesized audio data is found, a response message generated based on the pre-synthesized audio data is sent to the terminal device. If the pre-synthesized audio data is not found, audio synthesized data obtained after performing a TTS operation on the text response data is obtained and stored in the audio cache. By dynamically updating the cached audio synthesized data, the device can quickly respond to user requests during voice interaction, improving cache hit rate and timely playback of relevant audio. This dynamic cache update strategy shortens interaction time by trading space for time, providing a more efficient interactive experience and enhancing the real-time performance of human-computer interaction. It solves the technical problem of not being able to perform speech synthesis quickly and accurately, achieving not only fast and accurate speech synthesis but also improving the processing efficiency of user interaction requests.

[0089] In an exemplary embodiment, the lookup module is further configured to perform a TTS pre-synthesis operation on the text response data before obtaining the audio pre-synthesis data obtained after performing a TTS pre-synthesis operation on the text response data, thereby obtaining audio pre-synthesis data. Specific steps include: calling a preset audio synthesis service to perform a TTS pre-synthesis operation on the text response data and obtaining the audio pre-synthesis data generated after the TTS pre-synthesis operation; and pre-caching the audio pre-synthesis data in the audio cache library.

[0090] In an exemplary embodiment, the acquisition module is further configured to: parse the text data using a natural language processing model to obtain interaction intent keywords and determine the intent type corresponding to the interaction intent keywords; acquire intent data corresponding to the intent type from the database, and determine the text response data based on the intent data and the interaction intent keywords.

[0091] In an exemplary embodiment, the acquisition module is further configured to: after determining the text response data based on the intent data and the interaction intent keywords, further perform a data correction operation on the text response data to obtain corrected data; and determine the corrected data as the text response data, wherein the data correction operation includes at least one of the following: a scene replacement operation, or a script update operation.

[0092] In an exemplary embodiment, the data correction operation includes a script update operation. The acquisition module is further configured to: perform a data correction operation on the text response data to obtain corrected data. Specifically, the steps are as follows: perform a script update operation on the text response data to obtain corrected data by: modifying the text description script of the text response data using a default text description script to obtain corrected data; or, for a pre-launch database, acquire the pre-launch script of the pre-launch database, and modify the text description script of the text response data using the pre-launch script to obtain corrected data.

[0093] In an exemplary embodiment, the data correction operation includes a scene replacement operation. The acquisition module is further configured to: perform a data correction operation on the text response data to obtain corrected data. Specifically, the scene replacement operation on the text response data is performed in the following manner: when it is determined that the intent scene corresponding to the intent type has been updated, the intent data is updated synchronously; the text response data is determined based on the updated intent data and the interaction intent keyword, and a data correction operation is performed on the text response data to obtain corrected data.

[0094] In one exemplary embodiment, the speech synthesis apparatus further includes: a pre-caching module, configured to determine hot-spot interaction requests from interaction log records, wherein the hot-spot interaction requests include at least a first interaction request and a second interaction request, the request frequency of the first interaction request is higher than that of a first interaction request with a preset frequency, and the second audio pre-synthesized data corresponding to the second interaction request is not found in the audio cache library; to perform aggregation processing on the first audio pre-synthesized data corresponding to the first interaction request to obtain a first aggregation result, and to perform aggregation processing on the second audio pre-synthesized data to obtain a second aggregation result; and to pre-cachate the first aggregation result and the second aggregation result.

[0095] Embodiments of this application also provide a storage medium including a stored program, wherein the program executes any of the methods described above when it is run.

[0096] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0097] S1, convert the voice packet in the terminal interaction request from the terminal device into text data, and obtain the text response data indicated by the parsing result of the text data;

[0098] S2, Obtain audio pre-synthesized data after performing a text-to-speech (TTS) pre-synthesis operation on the text response data;

[0099] S3, if the audio pre-synthesized data is found, the response information generated based on the audio pre-synthesized data is sent to the terminal device;

[0100] S4, if the audio pre-synthesized data is not found, obtain the audio synthesized data obtained after performing TTS operation on the text response data, and store the audio synthesized data in the audio cache library.

[0101] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0102] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0103] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0104] S1, convert the voice packet in the terminal interaction request from the terminal device into text data, and obtain the text response data indicated by the parsing result of the text data;

[0105] S2, Obtain audio pre-synthesized data after performing a text-to-speech (TTS) pre-synthesis operation on the text response data;

[0106] S3, if the audio pre-synthesized data is found, the response information generated based on the audio pre-synthesized data is sent to the terminal device;

[0107] S4, if the audio pre-synthesized data is not found, obtain the audio synthesized data obtained after performing TTS operation on the text response data, and store the audio synthesized data in the audio cache library.

[0108] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0109] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0110] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0111] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A speech synthesis method characterized by, The method comprises the following steps: Converting a voice packet in a terminal interaction request from a terminal device into text data, and obtaining text response data indicated by a parsing result of the text data; Obtaining audio pre-synthesis data obtained by performing a text-to-speech (TTS) pre-synthesis operation on the text response data; In a case where the audio pre-synthesis data is found, sending response information generated based on the audio pre-synthesis data to the terminal device; In a case where the audio pre-synthesis data is not found, obtaining audio synthesis data obtained by performing a TTS synthesis operation on the text response data, and storing the audio synthesis data into an audio cache library; The method further comprises the following steps: Determining hot spot interaction requests from interaction logs, wherein the hot spot interaction requests at least include a first interaction request and a second interaction request, the request frequency of the first interaction request is higher than that of a first interaction request with a preset frequency, and second audio pre-synthesis data corresponding to the second interaction request is not found in the audio cache library; Performing aggregation processing on first audio pre-synthesis data corresponding to the first interaction request to obtain a first aggregation result, and performing aggregation processing on the second audio pre-synthesis data to obtain a second aggregation result; Pre-caching the first aggregation result and the second aggregation result; Before obtaining the audio pre-synthesis data obtained by performing the TTS pre-synthesis operation on the text response data, the method further comprises the following steps: Calling a preset audio synthesis service to perform the TTS pre-synthesis operation on the text response data, and obtaining the audio pre-synthesis data generated after the TTS pre-synthesis operation; Pre-caching the audio pre-synthesis data into the audio cache library.

2. The speech synthesis method of claim 1, wherein, Obtaining the text response data indicated by the parsing result of the text data comprises the following steps: Using a natural language processing model to parse the text data to obtain an interaction intent keyword, and determining an intent type corresponding to the interaction intent keyword; Obtaining intent data corresponding to the intent type from a database, and determining the text response data according to the intent data and the interaction intent keyword.

3. The speech synthesis method of claim 2, wherein, After determining the text response data according to the intent data and the interaction intent keyword, the method further comprises the following steps: Performing a data correction operation on the text response data to obtain corrected data; Determining the corrected data as the text response data, wherein the data correction operation at least includes one of the following: a replacement scenario operation and a script updating operation.

4. The speech synthesis method of claim 3, wherein, The data correction operation includes a script updating operation, and performing a data correction operation on the text response data to obtain corrected data comprises the following steps: Performing a data correction operation on the text response data to obtain corrected data comprises the following steps: Performing a script updating operation on the text response data to obtain corrected data by the following method: Modifying a text description script of the text response data using a default text description script to obtain corrected data; Or, for a pre-online database, obtain a pre-online dialogue of the pre-online database, modify the text description dialogue of the text response data using the pre-online dialogue, and obtain modified data.

5. The speech synthesis method of claim 3, wherein, The data modification operation includes a replacement scenario operation, and the text response data is subjected to the data modification operation to obtain modified data, including: The text response data is subjected to the data modification operation to obtain modified data, including: The text response data is subjected to the replacement scenario operation in the following manner: In a case where it is determined that the intent scene corresponding to the intent type is updated, the intent data is synchronously updated; According to the updated intent data and the interactive intent keyword, the text response data is determined, and the text response data is subjected to a data modification operation to obtain modified data.

6. A speech synthesis apparatus characterized by comprising: Including: The acquisition module is configured to convert a voice packet in a terminal interaction request from a terminal device into text data, and acquire text response data indicated by a parsing result of the text data; The finding module is configured to acquire audio pre-synthesis data obtained by performing a text-to-speech (TTS) pre-synthesis operation on the text response data; The sending module is configured to, in a case where the audio pre-synthesis data is found, send response information generated based on the audio pre-synthesis data to the terminal device; The storage module is configured to, in a case where the audio pre-synthesis data is not found, acquire audio synthesis data obtained by performing a TTS operation on the text response data, and store the audio synthesis data in an audio cache library; The finding module is further configured to, before acquiring the audio pre-synthesis data obtained by performing the TTS pre-synthesis operation on the text response data, invoke a preset audio synthesis service to perform the TTS pre-synthesis operation on the text response data, and obtain the audio pre-synthesis data generated after the TTS pre-synthesis operation; and pre-cache the audio pre-synthesis data in the audio cache library; The pre-caching module is configured to determine hotpoint interaction requests from interaction log records, wherein the hotpoint interaction requests at least include a first interaction request and a second interaction request, the request frequency of the first interaction request is higher than that of a first interaction request with a preset frequency, and second audio pre-synthesis data corresponding to the second interaction request is not found in the audio cache library. The first audio pre-synthesis data corresponding to the first interaction request is subjected to aggregation processing to obtain a first aggregation result, and the second audio pre-synthesis data is subjected to aggregation processing to obtain a second aggregation result; and the first aggregation result and the second aggregation result are pre-cached.

7. A computer readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 5 when running. 8.An electronic device comprising a memory and a processor, the electronic device comprising: The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 5 by using the computer program.

Images