Speech synthesis method and device, storage medium and electronic device

Abstract

The invention discloses a voice synthesis method and device, a storage medium and an electronic device, and relates to the technical field of smart home, and the method comprises the steps: converting a voice packet in a terminal interaction request from terminal equipment into text data, and obtaining text response data indicated by an analysis result of the text data; obtaining audio pre-synthesis data obtained after TTS pre-synthesis operation is carried out on the text response data; under the condition that the audio pre-synthesis data is found, sending response information generated based on the audio pre-synthesis data to the terminal equipment; and under the condition that the audio pre-synthesis data is not found, obtaining the audio synthesis data obtained after the TTS synthesis operation is performed on the text response data, and storing the audio synthesis data in the audio cache library, and by adopting the technical scheme, the technical problem that speech synthesis cannot be performed quickly and accurately is solved.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and more specifically, to a speech synthesis method, device, storage medium and electronic device. Background Art

[0002] At present, with the rapid development of artificial intelligence technology, especially the advancement of artificial intelligence large models, human-computer voice interaction is increasingly widely used in the field of smart homes. This interactive method provides users with a more natural and convenient operating experience through media such as voice. However, real-time audio synthesis is required during voice interaction, and the response speed is not high. In related technologies, in order to improve the response speed, a strategy of caching synthesized voice data is adopted. The basic idea of ​​this strategy is to generate and store some commonly used voice data in advance so that a response can be provided quickly when the user requests it. However, the data pre-stored in this method is difficult to cover the actual interaction needs of users, and the hit rate is not high, which greatly reduces the effectiveness of the cache and cannot quickly and accurately perform voice synthesis.

[0003] Therefore, in the related art, there is a technical problem that speech synthesis cannot be performed quickly and accurately.

[0004] With respect to the technical problem in the related art that speech synthesis cannot be performed quickly and accurately, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a speech synthesis method, apparatus, storage medium and electronic device to at least solve the technical problem in the related art that speech synthesis cannot be performed quickly and accurately.

[0006] According to an embodiment of the present application, a speech synthesis method is provided, including: converting a voice packet in a terminal interaction request from a terminal device into text data, and obtaining text response data indicated by a parsing result of the text data; obtaining audio pre-synthesized data obtained after performing a text-to-speech TTS pre-synthesis operation on the text response data; when the audio pre-synthesized data is found, sending response information generated based on the audio pre-synthesized data to the terminal device; when the audio pre-synthesized data is not found, obtaining audio synthesis data obtained after performing a TTS operation on the text response data, and storing the audio synthesis data in the audio cache.

[0007] In an exemplary embodiment, the method also includes: determining hot spot interaction requests from interaction log records, wherein the hot spot interaction requests include at least a first interaction request and a second interaction request, the request frequency of the first interaction request is higher than the first interaction request of a preset frequency, and the second audio pre-synthesis data corresponding to the second interaction request is not found in the audio cache library; aggregating the first audio pre-synthesis data corresponding to the first interaction request to obtain a first aggregation result, and aggregating the second audio pre-synthesis data to obtain a second aggregation result; and pre-caching the first aggregation result and the second aggregation result.

[0008] In an exemplary embodiment, before obtaining the audio pre-synthesis data obtained after performing a TTS pre-synthesis operation on the text response data, the method also includes: calling a preset audio synthesis service to perform a TTS pre-synthesis operation on the text response data, and obtaining the audio pre-synthesis data generated after the TTS pre-synthesis operation; and pre-caching the audio pre-synthesis data into the audio cache library.

[0009] In an exemplary embodiment, obtaining text response data indicated by the parsing result of the text data includes: parsing the text data using a natural language processing model to obtain interaction intent keywords, and determining the intent type corresponding to the interaction intent keyword; obtaining intent data corresponding to the intent type from a database, and determining the text response data based on the intent data and the interaction intent keyword.

[0010] In an exemplary embodiment, after determining the text response data based on the intent data and the interaction intent keywords, the method further includes: performing a data correction operation on the text response data to obtain corrected data; determining the corrected data as the text response data, wherein the data correction operation includes at least one of the following: a scene replacement operation and a speech update operation.

[0011] In an exemplary embodiment, the data correction operation includes a speech updating operation, and a data correction operation is performed on the text response data to obtain the corrected data, including: performing a data correction operation on the text response data to obtain the corrected data, including: performing a speech updating operation on the text response data to obtain the corrected data in the following manner: using a default text description speech to modify the text description speech of the text response data to obtain the corrected data; or, for a pre-online database, obtaining the pre-online speech of the pre-online database, and using the pre-online speech to modify the text description speech of the text response data to obtain the corrected data.

[0012] In an exemplary embodiment, the data correction operation includes a scene replacement operation, performing a data correction operation on the text response data to obtain corrected data, including: performing a data correction operation on the text response data to obtain corrected data, including: performing a scene replacement operation on the text response data in the following manner: when it is determined that the intent scene corresponding to the intent type is updated, synchronously updating the intent data; determining the text response data based on the updated intent data and the interactive intent keyword, performing a data correction operation on the text response data to obtain corrected data.

[0013] According to another aspect of an embodiment of the present application, a speech synthesis device is also provided, including: an acquisition module, used to convert a voice packet in a terminal interaction request from a terminal device into text data, and obtain text response data indicated by the parsing result of the text data; a search module, used to obtain audio pre-synthesized data obtained after performing a text-to-speech TTS pre-synthesis operation on the text response data; a sending module, used to send response information generated based on the audio pre-synthesized data to the terminal device when the audio pre-synthesized data is found; and a storage module, used to obtain the audio synthesis data obtained after performing a TTS operation on the text response data when the audio pre-synthesized data is not found, and store the audio synthesis data in the audio cache.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned speech synthesis method when running.

[0015] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the speech synthesis method through the computer program.

[0016] In an embodiment of the present application, by converting a voice packet in a terminal interaction request from a terminal device into text data, and obtaining text response data indicated by the parsing result of the text data; obtaining audio pre-synthesized data obtained after performing a text-to-speech TTS pre-synthesis operation on the text response data; in the case where the audio pre-synthesized data is found, sending the response information generated based on the audio pre-synthesized data to the terminal device; in the case where the audio pre-synthesized data is not found, obtaining the audio synthesis data obtained after performing a TTS operation on the text response data, and storing the audio synthesis data in the audio cache library; adopting the above technical solution, by dynamically updating the audio synthesis data in the cache, the cached audio synthesis data can be used to quickly respond to user requests when the user performs voice interaction, which not only improves the cache hit rate, but also can timely broadcast the corresponding audio to the user. This dynamic update cache strategy shortens the interaction time by exchanging space for time, provides a more efficient interaction experience, and improves the real-time nature of human-computer interaction. The technical problem of not being able to quickly and accurately perform speech synthesis is solved, not only fast and accurate speech synthesis is achieved, but also the processing efficiency of user interaction requests is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 It is a schematic diagram of the hardware environment of a speech synthesis method according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of a speech synthesis method according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an implementation process of a speech synthesis method according to an embodiment of the present application;

[0022] Figure 4 It is a structural block diagram of a speech synthesis device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] According to one aspect of an embodiment of the present application, a speech synthesis method is provided. The speech synthesis method is widely used in smart home (Smart Home), smart home, smart home device ecology, smart house (Intelligence House) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned speech synthesis method can be applied to Figure 1 In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0026] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart fresh air device, a smart kitchen and bathroom device, a smart bathroom device, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0027] In this embodiment, a speech synthesis method is provided, which is applied to the above terminal device. Figure 2 : is a flow chart of a speech synthesis method according to an embodiment of the present application, the flow chart includes the following steps:

[0028] Step S202, converting a voice packet in a terminal interaction request from a terminal device into text data, and acquiring text response data indicated by a parsing result of the text data;

[0029] The analysis of text data is understood as, for example, performing intent analysis on the text data.

[0030] Use the NLP (Natural Language Processing) model for intent analysis. Intent analysis is an important task in the field of natural language processing. It aims to identify and understand the intention behind the natural language text input by the user. In simple terms, it enables the computer to understand what operation the user wants to perform or what question the user wants to ask.

[0031] Step S204, obtaining audio pre-synthesis data obtained by performing a text-to-speech (TTS) pre-synthesis operation on the text response data;

[0032] The above-mentioned TTS pre-synthesis operation may be understood as TTS pre-synthesis, for example.

[0033] In this application, TTS (Text-to-Speech) is a technology that converts text information into audible speech, which can improve the performance of speech synthesis applications and user experience. TTS technology is widely used in various scenarios, such as voice assistants, e-book readers, navigation systems, etc. In the context of TTS real-time caching, TTS refers to real-time generated speech data.

[0034] A cache is a storage mechanism that temporarily saves data for quick access. Therefore, TTS real-time cache means a cache that stores real-time generated speech data so that it can be played or processed quickly when needed. This cache mechanism can improve the responsiveness and efficiency of speech synthesis.

[0035] Among them, TTS pre-synthesis means converting text into voice files and saving them in advance before voice output is actually needed. In this way, when users need to hear these voices, the system can directly play the pre-synthesized audio without the need for real-time text-to-speech conversion. Pre-synthesis can reduce the delay of real-time processing and improve response speed, especially in poor network conditions or limited computing resources.

[0036] TTS pre-caching means storing pre-synthesized voice files locally or on a server for quick access when needed. Pre-caching can reduce network latency and improve the smoothness of voice playback. For example, in a voice assistant app, developers may pre-synthesize some commonly used phrases or commands and cache these audio files on the user's device. When the user speaks these phrases, the app can play the cached audio directly without waiting for network requests or real-time synthesis.

[0037] By combining TTS pre-synthesis and TTS pre-caching, the performance and user experience of TTS applications can be significantly improved by preparing and storing voice data in advance, reducing the need for real-time processing, thereby improving response speed and user experience. For example, in an online reading application, developers may pre-synthesize chapters that users may read and directly play these audios when users read, rather than synthesizing them in real time. This can reduce waiting time and improve reading coherence.

[0038] Step S206, when the audio pre-synthesis data is found, sending response information generated based on the audio pre-synthesis data to the terminal device;

[0039] Optionally, for example, the audio pre-synthesis data is searched from an audio cache library.

[0040] Step S208: If the pre-synthesized audio data is not found, obtain the synthesized audio data obtained by performing a TTS operation on the text response data, and store the synthesized audio data in an audio cache.

[0041] In this step, the TTS synthesis operation is, for example, requesting a self-developed TTS algorithm or a third-party TTS service to perform TTS audio synthesis. Optionally, in this step, during voice interaction, the corresponding text can be obtained through the interface of data providers such as weather, news, knowledge base, and music, and then the TTS speech synthesis service (i.e., the self-developed TTS algorithm or the TTS speech synthesis service provided by a third party) can be used for audio synthesis, and the synthesized audio can be played.

[0042] Through the above steps, the voice packet in the terminal interaction request from the terminal device is converted into text data, and the text response data indicated by the parsing result of the text data is obtained; the audio pre-synthesized data obtained after the text-to-speech TTS pre-synthesis operation is performed on the text response data; when the audio pre-synthesized data is found, the response information generated based on the audio pre-synthesized data is sent to the terminal device; when the audio pre-synthesized data is not found, the audio synthesis data obtained after the TTS operation is performed on the text response data, and the audio synthesis data is stored in the audio cache library; adopting the above technical solution, by dynamically updating the audio synthesis data in the cache, the cached audio synthesis data can be used to quickly respond to the user request when the user performs voice interaction, which not only improves the cache hit rate, but also can timely broadcast the corresponding audio to the user. This dynamic update cache strategy shortens the interaction time in a space-for-time manner, provides a more efficient interaction experience, improves the real-time nature of human-computer interaction, solves the technical problem that speech synthesis cannot be performed quickly and accurately, and not only realizes fast and accurate speech synthesis, but also further improves the processing efficiency of user interaction requests.

[0043] In an exemplary embodiment, before obtaining the audio pre-synthesis data obtained after performing a TTS pre-synthesis operation on the text response data, further, a preset audio synthesis service can be called to perform a TTS pre-synthesis operation on the text response data, and obtain the audio pre-synthesis data generated after the TTS pre-synthesis operation; and pre-cache the audio pre-synthesis data into the audio cache library.

[0044] In this embodiment, the text response data can be pre-cached by TTS during the user voice interaction process. At this time, since the terminal and other business scenarios may change the text response data, the pre-cached text response data is not the actual synthesized TTS final text. For example, the matching degree between the pre-cached text response data and the final text is between 60% and 70%.

[0045] Optionally, in one embodiment, during the user interaction process, if the audio pre-synthesized data is not found in the audio cache library, it is a cache miss, and the audio pre-synthesized data is immediately cached, for example, the cache validity period is 1 hour. If the audio pre-synthesized data is found in the audio cache library, it is a cache hit, and the cache validity period is reset, for example, if the original 1-day validity period now expires in 1 hour, the validity period is reset to 1 day.

[0046] In an exemplary embodiment, the implementation process of obtaining text response data indicated by the parsing result of the text data may include: using a natural language processing model to parse the text data to obtain interaction intent keywords, and determining the intent type corresponding to the interaction intent keywords; obtaining intent data corresponding to the intent type from a database, and determining the text response data based on the intent data and the interaction intent keywords.

[0047] In an exemplary embodiment, after determining the text response data based on the intent data and the interaction intent keywords, further, a data correction operation is performed on the text response data to obtain corrected data; the corrected data is determined as the text response data, wherein the data correction operation includes at least one of the following: a scene replacement operation and a speech update operation.

[0048] In an exemplary embodiment, the data correction operation includes a wording update operation, and the technical solution of performing a data correction operation on the text response data to obtain the corrected data specifically includes: performing a data correction operation on the text response data to obtain the corrected data, and the specific steps are: performing a wording update operation on the text response data to obtain the corrected data in the following manners: using the default text description words to modify the text description words of the text response data to obtain the corrected data; or, for a pre-online database, obtaining the pre-online words of the pre-online database, and using the pre-online words to modify the text description words of the text response data to obtain the corrected data.

[0049] In an exemplary embodiment, the data correction operation includes a scene replacement operation, and the data correction operation is performed on the text response data to obtain the corrected data. The implementation process includes: performing a data correction operation on the text response data to obtain the corrected data, specifically performing a scene replacement operation on the text response data in the following manner: when it is determined that the intent scene corresponding to the intent type is updated, the intent data is synchronously updated; the text response data is determined based on the updated intent data and the interactive intent keyword, and a data correction operation is performed on the text response data to obtain the corrected data. In this embodiment, by performing data correction on the text response data, real-time updating of the data can be achieved, thereby improving the accuracy of the data.

[0050] In an exemplary embodiment, a technical solution is also proposed, which specifically includes: determining hot spot interaction requests from interaction log records, wherein the hot spot interaction requests include at least a first interaction request and a second interaction request, the request frequency of the first interaction request is higher than the first interaction request of a preset frequency, and the second audio pre-synthesis data corresponding to the second interaction request is not found in the audio cache library; aggregating the first audio pre-synthesis data corresponding to the first interaction request to obtain a first aggregation result, and aggregating the second audio pre-synthesis data to obtain a second aggregation result; and pre-caching the first aggregation result and the second aggregation result.

[0051] Optionally, in one embodiment, the cache validity period, cache quantity and cache size of the audio pre-synthesized data can be dynamically adjusted according to the timbre, volume, speech speed and other attributes of the audio pre-synthesized data to manage and maintain the cache content.

[0052] In addition, the present application can perform personalized and fine-grained differentiated management of cache data based on user interaction requests, effectively improving the cache hit rate and reducing the storage pressure of speech synthesis and cache.

[0053] In order to better understand the process of the above-mentioned speech synthesis method, the implementation method flow of the above-mentioned speech synthesis is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0054] In this embodiment, a speech synthesis method is provided. Figure 3 is a schematic diagram of the implementation process of the speech synthesis method according to an embodiment of the present application, such as Figure 3 As shown, after the terminal picks up the sound, the cloud ASR recognizes the audio to obtain text, performs NLP intent analysis on the text, and obtains the content text (i.e. the above-mentioned text response data) based on the parsed intent, and obtains the corrected and changed final content text for TTS audio synthesis.

[0055] Among them, the accurate acquisition of content text reflects the real-time nature of the data (it cannot be old data). Whether the TTS audio synthesis is correct and whether the synthesis is time-consuming will affect the subsequent cache hit accuracy. This application improves the interactive experience through two key links: accurately obtaining content text and increasing the speed of TTS audio synthesis.

[0056] like Figure 3 As shown, in this application, it includes TTS real-time cache, TTS pre-cache, TTS hot data cache, fixed speech cache and pre-launch speech cache. The validity period of these types of caches can be configured and managed.

[0057] Among them, real-time TTS cache means that during the user interaction process, if the current TTS synthesis does not hit the cache, it will be cached immediately after the TTS speech synthesis. Or if the current TTS synthesis hits the cache, the cache validity period corresponding to this article will be reset.

[0058] TTS pre-caching means that during the user voice interaction process, after voice recognition and intent recognition analysis, the acquired text content (without modification or change) is immediately pre-synthesized and cached using TTS (the matching degree between the content text result and the actual synthesized TTS text at this time is between 60% and 70%).

[0059] The hot data cache may include data that missed the cache and requested hot data, which represents hot data (including synthesized text, corresponding intent domain, timbre, speaking speed, volume, pitch, etc.) collected from user interaction data according to hourly, daily, and weekly time dimensions, and pre-cached the hot data based on text domain, timbre and other characteristics at different time dimensions (may include topN data that missed the cache and topN data requested by TTS hot spots).

[0060] Fixed scripts (corresponding to the default text description scripts) cache means caching the fixed scripts (which have a long usage period and will not be easily changed) defined according to various demand scenarios after audio synthesis. The cache period can be set to six months to one year, and dynamic online and offline are supported.

[0061] Pre-launch script caching means pre-synthesizing and caching the audio for the pre-launch script set by the pre-launch function. For example, if you want to launch a knowledge base, perform TTS synthesis on the text of the knowledge base and cache it in advance.

[0062] Optionally, in one embodiment, the implementation process for TTS real-time caching and TTS pre-caching is as follows:

[0063] Step 1: The user initiates voice interaction through smart home devices (speakers, refrigerators, air conditioners, water heaters, and kitchen appliances) or device software, and sends audio data packets to the AI ​​cloud service;

[0064] Step 2: The AI ​​cloud service decodes and recognizes the audio and parses the text data corresponding to the speech;

[0065] Step 3: The AI ​​cloud service performs NLP intent analysis on the text data to parse out the text intent and the corresponding intent field (corresponding intent type) and other data;

[0066] Step 4: Use the text intent and intent domain data to obtain the response text (equivalent to the above text response data) from the database, such as weather, music, news and other data from third-party services, and then proceed to step 5 (asynchronous) and step 6 (synchronous);

[0067] Step 5: asynchronously trigger TTS pre-synthesis of the response text and perform TTS pre-caching.

[0068] Step 6: Synchronously trigger the modification of the response text (different terminal devices will have different modification logic) and the scene speech replacement (involving relevant configurations such as the speech platform) to obtain the final content response text;

[0069] Step 7: Request the TTS synthesis service to perform TTS audio synthesis on the content text data of the final response.

[0070] If the audio hits the cache, the audio is directly retrieved from the cache and returned.

[0071] If the audio does not hit the cache, request the self-developed TTS algorithm or third-party TTS service to perform TTS audio synthesis, and cache the audio data synthesized by this TTS request in real time.

[0072] Optionally, in one embodiment, the process for TTS hotspot data caching, fixed speech caching and pre-launch caching is as follows:

[0073] Step 1: The TTS log statistics service performs data statistics and analysis to obtain hot data. Hot data refers to TTS request data counted hourly, daily, and weekly, including the TopN cache misses and the TopN TTS high-frequency requests. Steps 2 and 3 are triggered in parallel.

[0074] Step 2, such as Figure 3 As shown in the figure, after obtaining the log statistical analysis data of each time dimension, the TTS cache operation service aggregates the TopN high-frequency TTS requests, processes them into TopN data available to the TTS service according to the synthesized text, timbre, speaking speed, volume, etc., and triggers the pre-caching strategy of high-frequency TTS requests;

[0075] Step 3: After obtaining the log statistics analysis data of each time dimension, the TTS cache operation service aggregates the TopN TTS requests that miss the cache, processes them into the TopN data (synthesized text, timbre, speech speed, volume, etc.) available to the TTS service, and triggers the TTS miss cache pre-caching strategy;

[0076] Step 4: After the pre-caching strategies in steps 2 and 3 are triggered, the TTS service will be requested to perform the TTS strategy caching process. At this time, TTS audio synthesis and caching are performed according to the request for subsequent real TTS synthesis requests. The caching period is based on the configuration of each time dimension in the TTS cache operation service.

[0077] Step 5: The process of caching fixed words and pre-launch words is basically similar, except that the words come from different sources. Fixed words are basically fixed interactive response words within the scope of smart home, and pre-launch words are scene and feature words that are rolled out with project requirements. You can manually configure them through the TTS cache strategy operation platform.

[0078] Step 6: Based on step 5, the product configures the TTS pre-cache for the synchronously triggered speech, synchronously requests the TTS service, and proceeds to step 7.

[0079] Step 7: After receiving the request, the TTS service performs TTS audio synthesis. If the synthesized audio can hit the cache, the audio is directly retrieved from the cache and returned. If it does not hit, the self-developed TTS algorithm or third-party TTS service is used to synthesize TTS audio, and the audio data synthesized by this TTS request is cached in real time. The cache period can be manually configured according to the product in step 5.

[0080] Based on the above embodiments, on the one hand, dynamic strategies for real-time caching and pre-caching are proposed for different fields and different usage scenarios (fixed scripts, dynamic pre-launched scripts) such as weather, knowledge base, news, music, etc. For example: weather is only cached for 3 hours, knowledge base is cached for 1 week, news is cached for 1 day, etc., which can cope with the scenario when TTS requests suddenly increase, improve the cache hit rate when the same text is requested later, and improve the real-time TTS cache hit rate. On the other hand, TTS strategy caches for hot data, fixed scripts, and pre-launched scripts can cope with scenarios of general TTS requests and long-tail TTS requests, and improve the cache hit rate of periodic TTS requests.

[0081] In particular, the hot TTS cache strategy for TTS miss cache and hot high-frequency TTS requests can replenish cache data in time and improve the TTS cache hit rate. Therefore, this application can use the above cache strategies in combination according to different demand scenarios, improve the overall real-time and efficiency of TTS interaction during voice interaction, reduce the time and cost of TTS synthesis, and improve the voice interaction experience.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0083] Figure 4 is a structural block diagram of a speech synthesis device according to an embodiment of the present application; Figure 4 As shown, including:

[0084] An acquisition module 42, configured to convert a voice packet in a terminal interaction request from a terminal device into text data, and acquire text response data indicated by a parsing result of the text data;

[0085] A search module 44 is used to obtain audio pre-synthesis data obtained by performing a text-to-speech TTS pre-synthesis operation on the text response data;

[0086] A sending module 46, configured to send response information generated based on the audio presynthesis data to the terminal device when the audio presynthesis data is found;

[0087] The storage module 48 is used to obtain the audio synthesis data obtained after performing the TTS operation on the text response data when the audio pre-synthesis data is not found, and store the audio synthesis data in the audio cache.

[0088] Through the above-mentioned device, by converting the voice packet in the terminal interaction request from the terminal device into text data, and obtaining the text response data indicated by the parsing result of the text data; obtaining the audio pre-synthesized data obtained after the text-to-speech TTS pre-synthesis operation is performed on the text response data; in the case of finding the audio pre-synthesized data, sending the response information generated based on the audio pre-synthesized data to the terminal device; in the case of not finding the audio pre-synthesized data, obtaining the audio synthesis data obtained after the TTS operation is performed on the text response data, and storing the audio synthesis data in the audio cache library; adopting the above-mentioned technical scheme, by dynamically updating the audio synthesis data in the cache, the cached audio synthesis data can be used to quickly respond to the user request when the user performs voice interaction, which not only improves the cache hit rate, but also can timely broadcast the corresponding audio to the user. This dynamic update cache strategy shortens the interaction time by exchanging space for time, provides a more efficient interaction experience, and improves the real-time nature of human-computer interaction. The technical problem of not being able to quickly and accurately perform speech synthesis is solved, not only fast and accurate speech synthesis is achieved, but also the processing efficiency of user interaction requests is improved.

[0089] In an exemplary embodiment, the search module is further used to perform a TTS presynthesis operation on the text response data to obtain the audio presynthesis data obtained after the TTS presynthesis operation is performed on the text response data, and the specific steps include: calling a preset audio synthesis service to perform a TTS presynthesis operation on the text response data, and obtaining the audio presynthesis data generated after the TTS presynthesis operation; pre-caching the audio presynthesis data into the audio cache library;

[0090] In an exemplary embodiment, the acquisition module is also used to: use a natural language processing model to parse the text data to obtain interaction intent keywords, and determine the intent type corresponding to the interaction intent keywords; obtain intent data corresponding to the intent type from a database, and determine the text response data based on the intent data and the interaction intent keywords.

[0091] In an exemplary embodiment, the acquisition module is also used to: after determining the text response data based on the intent data and the interaction intention keyword, further perform a data correction operation on the text response data to obtain corrected data; determine the corrected data as the text response data, wherein the data correction operation includes at least one of the following: a scene replacement operation and a speech update operation.

[0092] In an exemplary embodiment, the data correction operation includes a wording update operation, and the acquisition module is also used to: perform a data correction operation on the text response data to obtain the corrected data, and the specific steps are: perform a wording update operation on the text response data to obtain the corrected data in the following manner: use the default text description words to modify the text description words of the text response data to obtain the corrected data; or, for a pre-online database, obtain the pre-online words of the pre-online database, and use the pre-online words to modify the text description words of the text response data to obtain the corrected data.

[0093] In an exemplary embodiment, the data correction operation includes a scene replacement operation, and the acquisition module is also used to: perform a data correction operation on the text response data to obtain corrected data, specifically by performing a scene replacement operation on the text response data in the following manner: when it is determined that the intent scene corresponding to the intent type is updated, the intent data is synchronously updated; the text response data is determined based on the updated intent data and the interactive intent keyword, and a data correction operation is performed on the text response data to obtain corrected data.

[0094] In an exemplary embodiment, the speech synthesis device also includes: a pre-caching module, used to determine hot spot interaction requests from interaction log records, wherein the hot spot interaction requests include at least a first interaction request and a second interaction request, the request frequency of the first interaction request is higher than the first interaction request of a preset frequency, and the second audio pre-synthesis data corresponding to the second interaction request is not found in the audio cache library; aggregating the first audio pre-synthesis data corresponding to the first interaction request to obtain a first aggregation result, and aggregating the second audio pre-synthesis data to obtain a second aggregation result; and pre-caching the first aggregation result and the second aggregation result.

[0095] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0096] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0097] S1, converting a voice packet in a terminal interaction request from a terminal device into text data, and obtaining text response data indicated by a parsing result of the text data;

[0098] S2, obtaining audio pre-synthesis data obtained by performing a text-to-speech (TTS) pre-synthesis operation on the text response data;

[0099] S3, when the audio pre-synthesis data is found, sending response information generated based on the audio pre-synthesis data to the terminal device;

[0100] S4, when the audio pre-synthesis data is not found, obtaining audio synthesis data obtained by performing a TTS operation on the text response data, and storing the audio synthesis data in an audio cache.

[0101] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0102] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0103] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0104] S1, converting a voice packet in a terminal interaction request from a terminal device into text data, and obtaining text response data indicated by a parsing result of the text data;

[0105] S2, obtaining audio pre-synthesis data obtained by performing a text-to-speech (TTS) pre-synthesis operation on the text response data;

[0106] S3, when the audio pre-synthesis data is found, sending response information generated based on the audio pre-synthesis data to the terminal device;

[0107] S4, when the audio pre-synthesis data is not found, obtaining audio synthesis data obtained by performing a TTS operation on the text response data, and storing the audio synthesis data in an audio cache.

[0108] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0109] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0110] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0111] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A speech synthesis method, characterized in that: include: Converting a voice packet in a terminal interaction request from a terminal device into text data, and acquiring text response data indicated by a parsing result of the text data; Acquire audio pre-synthesis data obtained after performing a text-to-speech (TTS) pre-synthesis operation on the text response data; In case the audio pre-synthesis data is found, sending response information generated based on the audio pre-synthesis data to the terminal device; In the case that the audio pre-synthesis data is not found, the audio synthesis data obtained after performing a TTS synthesis operation on the text response data is obtained, and the audio synthesis data is stored in an audio cache library.

2. The speech synthesis method according to claim 1, characterized in that: The method further comprises: Determine a hotspot interaction request from the interaction log record, wherein the hotspot interaction request includes at least a first interaction request and a second interaction request, the request frequency of the first interaction request is higher than the first interaction request of a preset frequency, and the second audio pre-synthesized data corresponding to the second interaction request is not found in the audio cache library; Aggregate the first audio pre-synthesized data corresponding to the first interaction request to obtain a first aggregation result, and aggregate the second audio pre-synthesized data to obtain a second aggregation result; The first aggregation result and the second aggregation result are pre-cached.

3. The speech synthesis method according to claim 1, characterized in that: Before obtaining the audio presynthesis data obtained after performing a TTS presynthesis operation on the text response data, the method further includes: Calling a preset audio synthesis service to perform a TTS pre-synthesis operation on the text response data, and obtaining the audio pre-synthesis data generated after the TTS pre-synthesis operation; The audio pre-synthesis data is pre-cached in the audio cache library.

4. The speech synthesis method according to claim 1, characterized in that: Acquiring text response data indicated by the parsing result of the text data, including: Parsing the text data using a natural language processing model to obtain interaction intent keywords, and determining the intent type corresponding to the interaction intent keywords; The intent data corresponding to the intent type is acquired from a database, and the text response data is determined according to the intent data and the interaction intent keyword.

5. The speech synthesis method according to claim 4, characterized in that: After determining the text response data according to the intention data and the interaction intention keyword, the method further includes: Performing a data correction operation on the text response data to obtain corrected data; The modified data is determined as the text response data, wherein the data modification operation includes at least one of the following: a scene replacement operation and a speech updating operation.

6. The speech synthesis method according to claim 5, characterized in that: The data correction operation includes a speech updating operation, and the data correction operation is performed on the text response data to obtain the corrected data, including: Performing a data correction operation on the text response data to obtain corrected data includes: Performing a speech updating operation on the text response data to obtain the corrected data in the following manner: modifying the text description speech of the text response data using a default text description speech to obtain the corrected data; Alternatively, for a pre-online database, the pre-online script of the pre-online database is obtained, and the text description script of the text response data is modified using the pre-online script to obtain modified data.

7. The speech synthesis method according to claim 5, characterized in that: The data correction operation includes a scene replacement operation, and the data correction operation is performed on the text response data to obtain the corrected data, including: Performing a data correction operation on the text response data to obtain corrected data includes: The text response data is subjected to a scene replacement operation in the following manner: When it is determined that the intent scenario corresponding to the intent type is updated, synchronously updating the intent data; The text response data is determined according to the updated intent data and the interactive intent keyword, and a data correction operation is performed on the text response data to obtain corrected data.

8. A speech synthesis device, characterized in that: include: An acquisition module, used to convert a voice packet in a terminal interaction request from a terminal device into text data, and acquire text response data indicated by a parsing result of the text data; A search module, used for obtaining audio pre-synthesis data obtained by performing a text-to-speech (TTS) pre-synthesis operation on the text response data; A sending module, configured to send response information generated based on the audio presynthesis data to the terminal device when the audio presynthesis data is found; The storage module is used to obtain the audio synthesis data obtained after performing TTS operation on the text response data when the audio pre-synthesis data is not found, and store the audio synthesis data in the audio cache library.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images