Text-based speech synthesis method, device, equipment and storage medium

Abstract

The present application relates to artificial intelligence technology, disclose a kind of text-based speech synthesis method, device, equipment and medium.The method comprises: optionally selecting one training data in financial training dataset;According to the data type of the training data, the speech-to-text module and the text-to-speech module in the initial speech synthesis model are determined as feedback module and labeling training module respectively, and the training data is converted into feedback data using the feedback module, the feedback data is converted into conversion data using the labeling training module, the initial speech synthesis model is adjusted based on the training data and the conversion data Return training data selection step until training is completed, and the target text is synthesized using the model.

Description

Technical Field

[0001] This invention relates to the fields of security detection technology and financial technology, and in particular to a text-based speech synthesis method, apparatus, electronic device and storage medium. Background Technology

[0002] In the financial services sector, to enhance users' understanding of relevant financial services and popularize related financial knowledge, relevant financial service texts are synthesized into speech and explained to users, facilitating their understanding of these services. For example, the requirements for applying for critical illness insurance can be synthesized into speech and explained to users, making it easier for them to understand the application requirements.

[0003] However, existing text-based speech synthesis methods require training ordinary text-to-speech models using labeled and aligned text and speech for speech synthesis. Since labeled and aligned text and speech are not easy to obtain, the training threshold for the model is high, resulting in low accuracy of speech synthesis. Summary of the Invention

[0004] This invention provides a text-based speech synthesis method, apparatus, electronic device, and storage medium, the main purpose of which is to improve the accuracy of speech synthesis of text required for handling insurance business such as critical illness insurance.

[0005] Obtain a training dataset of financial business knowledge, wherein each training data in the training dataset is training speech or training text;

[0006] Obtain an initial speech synthesis model, wherein the initial speech synthesis model includes: a speech-to-text module and a text-to-speech module;

[0007] Arbitrarily select one training data point from the financial business knowledge training dataset to obtain the target training data;

[0008] Based on the data type of the target training data, the speech-to-text module and the text-to-speech module are respectively determined as a feedback module and a labeling training module. The feedback module is used to perform data conversion on the target training data to obtain feedback data, and the labeling training module is used to perform data conversion on the feedback data to obtain converted data. The target training data and the converted data have the same data type, while the target training data and the feedback data have different data types, which are either text or speech.

[0009] The difference between the target training data and the transformed data is calculated using a pre-constructed loss function to obtain the target loss value;

[0010] When the target loss value is not less than the loss threshold, update the model parameters of the initial speech synthesis model and return to the step of arbitrarily selecting one training data from the financial business knowledge training dataset;

[0011] When the target loss value is less than the loss threshold, the initial speech synthesis model at this time is output to obtain the target speech synthesis model;

[0012] When a text containing financial business knowledge to be explained is received, the text-to-speech module in the target speech synthesis model is used to synthesize speech and obtain synthesized speech.

[0013] Optionally, the step of determining the speech-to-text module and the text-to-speech module as a feedback module and a labeled training module respectively according to the data type of the target training data, and using the feedback module to perform data conversion on the target training data to obtain feedback data, and using the labeled training module to perform data conversion on the feedback data to obtain converted data, includes:

[0014] When the data type of the target training data is text, the text-to-speech module is determined as the feedback module, and the speech-to-text module is determined as the labeling training module;

[0015] The feedback module is used to convert the target training data into speech form to obtain the feedback data;

[0016] The feedback data is converted into text using the labeled training module to obtain the converted data.

[0017] When the data type of the target training data is speech, the speech-to-text module is determined as the feedback module, and the text-to-speech module is determined as the label training module;

[0018] The feedback module is used to convert the target training data into text form to obtain the feedback data;

[0019] The feedback data is converted into speech using the labeled training module to obtain the converted data.

[0020] Optionally, the step of calculating the difference between the target training data and the transformed data using a pre-constructed loss function to obtain the target loss value includes:

[0021] Convert the target training data into target training vectors;

[0022] Convert the transformed data into a transformed data vector;

[0023] The difference between the target training vector and the transformed data vector is calculated using the loss function to obtain the target loss value.

[0024] Optionally, converting the target training data into target training vectors includes:

[0025] When the data type of the target training data is text,

[0026] The target training data is segmented into words to obtain one or more segmented words;

[0027] Each segmented word is mapped to a numerical vector to obtain the corresponding segmented word vector;

[0028] All the segmented word vectors are combined, and the combined vectors are then dimensionally compressed to obtain the target training vector.

[0029] When the data type of the target training data is speech,

[0030] The target training data is segmented into speech frames to obtain multiple speech frames;

[0031] Perform a Discrete Fourier Transform on each of the speech frames to convert the time-domain signal of each speech frame into a frequency-domain signal, thereby obtaining the frequency-domain feature information of each speech frame.

[0032] Vector feature transformation is performed based on the frequency domain feature information of each speech frame to obtain the speech frame vector of each speech frame;

[0033] Feature extraction is performed on all the aforementioned speech frame vectors to obtain the target training vector.

[0034] Optionally, the step of combining all the segmented word vectors and compressing the dimensionality of the combined vectors to obtain the target training vector includes:

[0035] Each segmented word vector is combined according to the order of the corresponding segmented words in the target training data to obtain a text vector sequence;

[0036] The target training vector is obtained by replacing each line in the text vector sequence with the average value of all elements in that line.

[0037] Optionally, the step of performing vector feature transformation based on the frequency domain feature information of each speech frame to obtain the speech frame vector of each speech frame includes:

[0038] Extract the amplitude spectrum from the frequency domain feature information of the speech frame;

[0039] The amplitude spectrum is moduloed or logarithmized to obtain the speech frame vector of the speech frame.

[0040] Optionally, updating the model parameters of the initial speech synthesis model includes:

[0041] The model parameters of the initial speech synthesis model are updated along the negative gradient direction of the target loss function.

[0042] To address the above problems, the present invention also provides a text-based speech synthesis device, the device comprising:

[0043] The data acquisition module is used to acquire a financial business knowledge training dataset, wherein each training data in the financial business knowledge training dataset is training speech or training text.

[0044] A model training module is used to obtain an initial speech synthesis model, which includes a speech-to-text module and a text-to-speech module. A training data set is arbitrarily selected from the financial business knowledge training dataset to obtain target training data. Based on the data type of the target training data, the speech-to-text module and the text-to-speech module are respectively designated as a feedback module and a labeled training module. The feedback module is used to convert the target training data to obtain feedback data, and the labeled training module is used to convert the feedback data to obtain converted data. The target training data and the converted data have the same data type, while the data types of the target training data and the feedback data are different, and the data type is either text or speech. A pre-constructed loss function is used to calculate the difference between the target training data and the converted data to obtain a target loss value. When the target loss value is not less than a loss threshold, the model parameters of the initial speech synthesis model are updated, and the step of arbitrarily selecting a training data set from the financial business knowledge training dataset is returned. When the target loss value is less than the loss threshold, the initial speech synthesis model at this time is output to obtain the target speech synthesis model.

[0045] The speech synthesis module is used to synthesize speech by using the text-to-speech module in the target speech synthesis model when receiving text containing financial business knowledge to be explained and popularized, thereby obtaining synthesized speech.

[0046] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0047] Memory, storing at least one computer program; and

[0048] The processor executes the computer program stored in the memory to implement the text-based speech synthesis method described above.

[0049] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the text-based speech synthesis method described above.

[0050] In this embodiment of the invention, one training data point is arbitrarily selected from the financial business knowledge training dataset to obtain the target training data;

[0051] Based on the data type of the target training data, the speech-to-text module and the text-to-speech module are respectively designated as a feedback module and a labeled training module. The feedback module is used to convert the target training data to obtain feedback data, and the labeled training module is used to convert the feedback data to obtain converted data. The target training data and the converted data have the same data type, while the data types of the target training data and the feedback data are different; the data type is either text or speech. A pre-constructed loss function is used to calculate the difference between the target training data and the converted data to obtain a target loss value. When the target loss value is not less than the loss threshold, the model parameters of the initial speech synthesis model are updated, and the arbitrarily selected financial business knowledge training data is returned. The training data step involves training a dataset; when the target loss value is less than the loss threshold, the initial speech synthesis model is output to obtain the target speech synthesis model; when receiving text containing financial business knowledge to be explained, the text-to-speech module in the target speech synthesis model is used for speech synthesis to obtain synthesized speech. When training the initial speech synthesis model, the speech-to-text module and the text-to-speech module are used for mutual supervision training. It does not require the use of labeled aligned text or speech, and the model can be trained using only unlabeled text and speech, which reduces the difficulty of model training and improves the convenience of speech synthesis. Therefore, the text-based speech synthesis method, device, electronic device, and readable storage medium proposed in this embodiment of the invention improve the accuracy of speech synthesis of text containing requirements for handling insurance business such as critical illness insurance. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a text-based speech synthesis method according to an embodiment of the present invention.

[0053] Figure 2 A schematic diagram of a text-based speech synthesis device provided in an embodiment of the present invention;

[0054] Figure 3 A schematic diagram of the internal structure of an electronic device implementing a text-based speech synthesis method according to an embodiment of the present invention;

[0055] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0056] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0057] This invention provides a text-based speech synthesis method. The execution entity of the text-based speech synthesis method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application embodiment: a server, a terminal, etc. In other words, the text-based speech synthesis method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0058] Reference Figure 1 The flowchart shown is a text-based speech synthesis method according to an embodiment of the present invention. In this embodiment, the text-based speech synthesis method includes the following steps:

[0059] S1. Obtain a financial business knowledge training dataset, wherein each training data in the financial business knowledge training dataset is training speech or training text.

[0060] In this embodiment of the invention, the training data is financial business knowledge data that can be used to synthesize text into a speech model. The training data in this embodiment can be training speech or training text, wherein all training speech is speech of financial business knowledge in the same language, and all training text is text of financial business knowledge in the same language, such as insurance business processing requirements text for various insurance businesses.

[0061] S2. Obtain an initial speech synthesis model, wherein the initial speech synthesis model includes: a speech-to-text module and a text-to-speech module;

[0062] Because training a single speech synthesis model requires a large amount of aligned speech and text, i.e., text requiring speech annotation for insurance business processing, the training effect of the model is poor when the amount of annotated data is small. To avoid this problem in this embodiment of the invention, an initial speech synthesis model is obtained. The initial speech synthesis model includes a speech-to-text module and a text-to-speech module. The speech-to-text module can convert speech into text, and the text-to-speech module can convert text into speech. By using the speech-to-text module to convert the speech into converted text, and then using the converted text as a pseudo-label for the speech, the model can be trained by measuring the degree of difference between the converted speech and the original speech. Alternatively, the text-to-speech module can be used to convert the text into converted speech, and then using the converted speech as a pseudo-label for the text, the model can be trained by measuring the degree of difference between the converted text and the original text. This not only enables training of the speech synthesis model with text without speech labels, but also allows training of the speech synthesis model with speech without text labels, improving the convenience and accuracy of model training.

[0063] S3. Randomly select one training data from the financial business knowledge training dataset to obtain the target training data;

[0064] In this embodiment of the invention, one training data point is arbitrarily selected from the financial business knowledge training dataset to obtain the target training data.

[0065] S4. Based on the data type of the target training data, the speech-to-text module and the text-to-speech module are respectively determined as a feedback module and a labeling training module. The feedback module is used to perform data conversion on the target training data to obtain feedback data. The labeling training module is used to perform data conversion on the feedback data to obtain converted data. The target training data and the converted data have the same data type, while the target training data and the feedback data have different data types. The data type is either text or speech.

[0066] In detail, in this embodiment of the invention, in order to perform labeled training on the target training data based on the feedback from the speech-to-text module and the text-to-speech module, the speech-to-text module and the text-to-speech module are respectively determined as a feedback module and a labeled training module according to the data type of the target training data. The feedback module is used to convert the target training data to obtain feedback data, and the labeled training module is used to convert the feedback data to obtain converted data. The target training data and the converted data have the same data type, while the target training data and the feedback data have different data types. The data type is either text or speech. When the target training data is training text, the data type is text; when the target training data is training speech, the data type is speech.

[0067] In detail, in this embodiment of the invention, the speech-to-text module and the text-to-speech module are respectively determined as a feedback module and a labeled training module according to the data type of the target training data. The feedback module is used to perform data conversion on the target training data to obtain feedback data, and the labeled training module is used to perform data conversion on the feedback data to obtain converted data. This includes:

[0068] When the data type of the target training data is text, the text-to-speech module is determined as the feedback module, and the speech-to-text module is determined as the labeling training module;

[0069] The feedback module is used to convert the target training data into speech form to obtain the feedback data;

[0070] The feedback data is converted into text using the labeled training module to obtain the converted data.

[0071] When the data type of the target training data is speech, the speech-to-text module is determined as the feedback module, and the text-to-speech module is determined as the label training module;

[0072] The feedback module is used to convert the target training data into text form to obtain the feedback data;

[0073] The feedback data is converted into speech using the labeled training module to obtain the converted data.

[0074] Optionally, the speech-to-text module in this embodiment of the invention is an adapted cascaded conformational model, which consists of a stream encoder, a delay encoder, and a HAT decoder. The stream encoder is a small convolutional neural network that converts speech frames into a 512-dimensional representation; the delay encoder is a large conformational neural network that converts the output of the stream encoder into a 640-dimensional representation; and the HAT decoder is an attention-based neural network that converts the output of the delay encoder into a text sequence.

[0075] The text-to-speech module is a model adapted from Tacotron 2, consisting of a text encoder and an audio decoder. The text encoder is a bidirectional LSTM neural network that converts text sequences into a 512-dimensional representation. The audio decoder is an autoregressive LSTM neural network that converts the output of the text encoder into audio samples. The audio decoder also includes a pre-network and a post-network for enhancing and processing the audio samples, respectively.

[0076] S5. Calculate the difference between the target training data and the transformed data using a pre-constructed loss function to obtain the target loss value;

[0077] In this embodiment of the invention, in order to measure the training progress of the model, the difference between the target training data and the transformed data is calculated using a pre-constructed loss function to obtain the target loss value.

[0078] In detail, in this embodiment of the invention, the difference between the target training data and the transformed data is calculated using a pre-constructed loss function to obtain the target loss value, including:

[0079] Convert the target training data into target training vectors;

[0080] Convert the transformed data into a transformed data vector;

[0081] The difference between the target training vector and the transformed data vector is calculated using the loss function to obtain the target loss value.

[0082] Optionally, the loss function described in the embodiments of the present invention includes, but is not limited to, the squared loss function, the absolute value loss function, the cross-entropy loss function, etc.

[0083] Furthermore, in this embodiment of the invention, converting the target training data into target training vectors includes:

[0084] When the data type of the target training data is text,

[0085] The target training data is segmented into words to obtain one or more segmented words;

[0086] Each segmented word is mapped to a numerical vector to obtain the corresponding segmented word vector;

[0087] All the segmented word vectors are combined, and the combined vectors are then dimensionally compressed to obtain the target training vector.

[0088] When the data type of the target training data is speech,

[0089] The target training data is segmented into speech frames to obtain multiple speech frames;

[0090] Perform a Discrete Fourier Transform on each of the speech frames to convert the time-domain signal of each speech frame into a frequency-domain signal, thereby obtaining the frequency-domain feature information of each speech frame.

[0091] Vector feature transformation is performed based on the frequency domain feature information of each speech frame to obtain the speech frame vector of each speech frame;

[0092] Feature extraction is performed on all the aforementioned speech frame vectors to obtain the target training vector.

[0093] Furthermore, in this embodiment of the invention, combining all the segmented word vectors and compressing the dimensions of the combined vectors to obtain the target training vector includes:

[0094] Each segmented word vector is combined according to the order of the corresponding segmented words in the target training data to obtain a text vector sequence;

[0095] The target training vector is obtained by replacing each line in the text vector sequence with the average value of all elements in that line.

[0096] In detail, the feature extraction of all the speech frame vectors to obtain the target training vector in this embodiment of the invention includes:

[0097] Each of the speech frame vectors is combined according to the sequential order of the corresponding speech frame in the target training data to obtain a speech vector sequence;

[0098] The target training vector is obtained by replacing each row in the speech vector sequence with the average value of all elements in that row.

[0099] Specifically, in this embodiment of the invention, the process of performing speech framing on the target training data includes operations such as noise reduction, framing, and windowing. These are preprocessing operations for speech framing and will not be elaborated here.

[0100] Furthermore, in this embodiment of the invention, the frequency domain feature information includes amplitude spectrum and phase spectrum. Based on the frequency domain feature information of each speech frame, vector feature transformation is performed to obtain the speech frame vector of each speech frame.

[0101] The specific methods for vector feature transformation are not specifically limited in the embodiments of the present invention.

[0102] Optionally, in this embodiment of the invention, vector feature transformation is performed based on the frequency domain feature information of each speech frame to obtain the speech frame vector of each speech frame, including:

[0103] Extract the amplitude spectrum from the frequency domain feature information of the speech frame;

[0104] The amplitude spectrum is moduloed or logarithmized to obtain the speech frame vector of the speech frame.

[0105] In another embodiment of the present invention, the amplitude spectrum and phase spectrum in the frequency domain feature information of the speech frame can be extracted, and the amplitude spectrum and phase spectrum can be concatenated in sequence to obtain the speech frame vector.

[0106] Furthermore, the method for converting the transformed data into a transformed data vector in this embodiment of the invention is similar to the method for converting the target training data into a target training vector, and will not be described in detail here.

[0107] S5. Determine whether the target loss value is less than a preset loss threshold;

[0108] S6. When the target loss value is not less than the loss threshold, update the model parameters of the initial speech synthesis model and return to the step of arbitrarily selecting one training data from the financial business knowledge training dataset;

[0109] In this embodiment of the invention, if the target loss value is less than the loss threshold, it indicates that the model accuracy of the initial speech synthesis model has not yet reached the preset accuracy requirement. Therefore, the model parameters of the initial speech synthesis model are adjusted, and the process returns to the step of arbitrarily selecting one training data from the financial business knowledge training dataset.

[0110] Optionally, in this embodiment of the invention, the model parameters of the initial speech synthesis model are updated along the negative gradient direction of the target loss function.

[0111] Furthermore, in this embodiment of the invention, the model parameters of the initial speech synthesis model can be adjusted using a pre-built model update algorithm. The model update algorithm in this embodiment of the invention can be gradient descent, momentum method, etc., and this embodiment of the invention does not limit it.

[0112] S7. When the target loss value is less than the loss threshold, output the initial speech synthesis model at this time to obtain the target speech synthesis model;

[0113] In this embodiment of the invention, when the target loss value is not less than the preset loss value, it indicates that the model accuracy of the initial speech synthesis model has reached the preset accuracy requirement, and the initial speech synthesis model at this time is output to obtain the target speech synthesis model.

[0114] S8. When receiving a text containing financial business knowledge to be explained and popularized, the text-to-speech module in the target speech synthesis model is used to synthesize speech and obtain synthesized speech.

[0115] In this embodiment of the invention, when a text containing financial business knowledge to be explained is received, it needs to be converted into speech for explanation to the user. The text is input into the text-to-speech module of the target speech synthesis model to obtain the synthesized speech. The text containing the financial business knowledge to be explained is a text describing the requirements for handling insurance services such as critical illness insurance, which needs to be converted into speech. By synthesizing the text of the critical illness insurance application requirements into speech, a synthesized speech is obtained, which is then used to explain the requirements for handling critical illness insurance and other insurance services to the user.

[0116] Furthermore, in this embodiment of the invention, after using the text-to-speech module in the target speech synthesis model to perform speech synthesis and obtain synthesized speech, the synthesized speech is sent to a preset terminal device, which can be a smart terminal such as a mobile phone, computer, or tablet.

[0117] like Figure 2 The diagram shown is a functional block diagram of the text-based speech synthesis device of the present invention.

[0118] The text-based speech synthesis device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the text-based speech synthesis device may include a data acquisition module 101, a model training module 102, and a speech synthesis module 103. The module mentioned in the present invention may also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0119] In this embodiment, the functions of each module / unit are as follows:

[0120] The data acquisition module 101 is used to acquire a financial business knowledge training dataset, wherein each training data in the financial business knowledge training dataset is training speech or training text.

[0121] The model training module 102 is used to obtain an initial speech synthesis model, wherein the initial speech synthesis model includes: a speech-to-text module and a text-to-speech module; arbitrarily select one training data from the financial business knowledge training dataset to obtain target training data; determine the speech-to-text module and the text-to-speech module as a feedback module and a labeled training module respectively according to the data type of the target training data, and use the feedback module to perform data conversion on the target training data to obtain feedback data, and use the labeled training module to perform data conversion on the feedback data to obtain converted data, wherein the data type of the target training data and the converted data are the same, and the data type of the target training data and the feedback data are different, the data type being text type or speech type; calculate the difference between the target training data and the converted data using a pre-constructed loss function to obtain a target loss value; when the target loss value is not less than the loss threshold, update the model parameters of the initial speech synthesis model and return to the step of arbitrarily selecting one training data from the financial business knowledge training dataset; when the target loss value is less than the loss threshold, output the initial speech synthesis model at this time to obtain the target speech synthesis model;

[0122] The speech synthesis module 103 is used to synthesize speech by using the text-to-speech module in the target speech synthesis model when it receives a text of financial business knowledge to be explained and popularized, so as to obtain synthesized speech.

[0123] In detail, the modules in the text-based speech synthesis device 100 described in this embodiment of the invention employ the same methods as described above. Figure 1 The text-based speech synthesis method described herein uses the same technical means and can produce the same technical effect, so it will not be repeated here.

[0124] like Figure 3 The diagram shown is a structural schematic of an electronic device that implements the text-based speech synthesis method of the present invention.

[0125] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a text-based speech synthesis program.

[0126] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as code for text-based speech synthesis programs, but also to temporarily store data that has been output or will be output.

[0127] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules (e.g., text-based speech synthesis programs) stored in the memory 11, and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0128] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The communication bus 12 is configured to enable communication between the memory 11 and at least one processor 10, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus.

[0129] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0130] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault classification circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0131] Optionally, the communication interface 13 may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device and other electronic devices.

[0132] Optionally, the communication interface 13 may further include a user interface, which may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0133] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0134] The text-based speech synthesis program stored in the memory 11 of the electronic device is a combination of multiple computer programs, which, when run in the processor 10, can achieve the following:

[0135] Obtain a training dataset of financial business knowledge, wherein each training data in the training dataset is training speech or training text;

[0136] Obtain an initial speech synthesis model, wherein the initial speech synthesis model includes: a speech-to-text module and a text-to-speech module;

[0137] Arbitrarily select one training data point from the financial business knowledge training dataset to obtain the target training data;

[0138] Based on the data type of the target training data, the speech-to-text module and the text-to-speech module are respectively determined as a feedback module and a labeling training module. The feedback module is used to perform data conversion on the target training data to obtain feedback data, and the labeling training module is used to perform data conversion on the feedback data to obtain converted data. The target training data and the converted data have the same data type, while the target training data and the feedback data have different data types, which are either text or speech.

[0139] The difference between the target training data and the transformed data is calculated using a pre-constructed loss function to obtain the target loss value;

[0140] When the target loss value is not less than the loss threshold, update the model parameters of the initial speech synthesis model and return to the step of arbitrarily selecting one training data from the financial business knowledge training dataset;

[0141] When the target loss value is less than the loss threshold, the initial speech synthesis model at this time is output to obtain the target speech synthesis model;

[0142] When a text containing financial business knowledge to be explained is received, the text-to-speech module in the target speech synthesis model is used to synthesize speech and obtain synthesized speech.

[0143] Specifically, the processor 10's implementation method of the above-mentioned computer program can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0144] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or volatile. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0145] Embodiments of the present invention may also provide a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0146] Obtain a training dataset of financial business knowledge, wherein each training data in the training dataset is training speech or training text;

[0147] Obtain an initial speech synthesis model, wherein the initial speech synthesis model includes: a speech-to-text module and a text-to-speech module;

[0148] Arbitrarily select one training data point from the financial business knowledge training dataset to obtain the target training data;

[0149] Based on the data type of the target training data, the speech-to-text module and the text-to-speech module are respectively determined as a feedback module and a labeling training module. The feedback module is used to perform data conversion on the target training data to obtain feedback data, and the labeling training module is used to perform data conversion on the feedback data to obtain converted data. The target training data and the converted data have the same data type, while the target training data and the feedback data have different data types, which are either text or speech.

[0150] The difference between the target training data and the transformed data is calculated using a pre-constructed loss function to obtain the target loss value;

[0151] When the target loss value is not less than the loss threshold, update the model parameters of the initial speech synthesis model and return to the step of arbitrarily selecting one training data from the financial business knowledge training dataset;

[0152] When the target loss value is less than the loss threshold, the initial speech synthesis model at this time is output to obtain the target speech synthesis model;

[0153] When a text containing financial business knowledge to be explained is received, the text-to-speech module in the target speech synthesis model is used to synthesize speech and obtain synthesized speech.

[0154] Furthermore, the computer's usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system, applications required for at least one function, etc.; and the data storage area may store data created based on the use of blockchain nodes, etc.

[0155] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0156] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0158] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0159] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0160] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0161] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0162] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A text-based speech synthesis method, characterized in that, The method includes: Obtain a training dataset of financial business knowledge, wherein each training data in the training dataset is training speech or training text; Obtain an initial speech synthesis model, wherein the initial speech synthesis model includes: a speech-to-text module and a text-to-speech module; Arbitrarily select one training data point from the financial business knowledge training dataset to obtain the target training data; Based on the data type of the target training data, the speech-to-text module and the text-to-speech module are respectively determined as a feedback module and a labeling training module. The feedback module is used to perform data conversion on the target training data to obtain feedback data, and the labeling training module is used to perform data conversion on the feedback data to obtain converted data. The target training data and the converted data have the same data type, while the target training data and the feedback data have different data types, which are either text or speech. The difference between the target training data and the transformed data is calculated using a pre-constructed loss function to obtain the target loss value; When the target loss value is not less than the loss threshold, update the model parameters of the initial speech synthesis model and return to the step of arbitrarily selecting one training data from the financial business knowledge training dataset; When the target loss value is less than the loss threshold, the initial speech synthesis model at this time is output to obtain the target speech synthesis model; When a text containing financial business knowledge to be explained and popularized is received, the text-to-speech module in the target speech synthesis model is used to synthesize speech and obtain synthesized speech. The step of calculating the difference between the target training data and the transformed data using a pre-constructed loss function to obtain the target loss value includes: converting the target training data into a target training vector; converting the transformed data into a transformed data vector; and calculating the difference between the target training vector and the transformed data vector using the loss function to obtain the target loss value. The step of converting the target training data into target training vectors includes: when the data type of the target training data is text, The target training data is segmented into words to obtain one or more segmented words; each segmented word is mapped to a numerical vector to obtain a corresponding segmented word vector; all segmented word vectors are combined, and the combined vector is dimensionally compressed to obtain the target training vector; when the data type of the target training data is speech, the target training data is segmented into speech frames to obtain multiple speech frames; a discrete Fourier transform is performed on each speech frame to convert the time-domain signal of each speech frame into a frequency-domain signal to obtain the frequency-domain feature information of each speech frame; vector feature transformation is performed based on the frequency-domain feature information of each speech frame to obtain the speech frame vector of each speech frame; feature extraction is performed on all speech frame vectors to obtain the target training vector.

2. The text-based speech synthesis method as described in claim 1, characterized in that, The step of determining the speech-to-text module and the text-to-speech module as a feedback module and a labeled training module respectively based on the data type of the target training data, and using the feedback module to perform data conversion on the target training data to obtain feedback data, and using the labeled training module to perform data conversion on the feedback data to obtain converted data, includes: When the data type of the target training data is text, the text-to-speech module is identified as the feedback module, and the speech-to-text module is identified as the labeling training module; The feedback module is used to convert the target training data into speech form to obtain the feedback data; The feedback data is converted into text using the labeled training module to obtain the converted data. When the data type of the target training data is speech, the speech-to-text module is determined as the feedback module, and the text-to-speech module is determined as the labeling training module; The feedback module is used to convert the target training data into text format to obtain the feedback data. The feedback data is converted into speech using the labeled training module to obtain the converted data.

3. The text-based speech synthesis method as described in claim 1, characterized in that, The step of combining all the segmented word vectors and compressing the dimensionality of the combined vectors to obtain the target training vector includes: Each segmented word vector is combined according to the order of the corresponding segmented words in the target training data to obtain a text vector sequence; The target training vector is obtained by replacing each line in the text vector sequence with the average value of all elements in that line.

4. The text-based speech synthesis method as described in claim 1, characterized in that, The step of performing vector feature transformation based on the frequency domain feature information of each speech frame to obtain the speech frame vector of each speech frame includes: Extract the amplitude spectrum from the frequency domain feature information of the speech frame; The amplitude spectrum is moduloed or logarithmized to obtain the speech frame vector of the speech frame.

5. The text-based speech synthesis method according to any one of claims 1 to 4, characterized in that, The updating of the model parameters of the initial speech synthesis model includes: The model parameters of the initial speech synthesis model are updated along the negative gradient direction of the target loss function.

6. A text-based speech synthesis apparatus for implementing the text-based speech synthesis method as described in any one of claims 1 to 5, characterized in that, include: The data acquisition module is used to acquire a financial business knowledge training dataset, wherein each training data in the financial business knowledge training dataset is training speech or training text. A model training module is used to obtain an initial speech synthesis model, wherein the initial speech synthesis model includes a speech-to-text module and a text-to-speech module; a training data set is arbitrarily selected from the financial business knowledge training dataset to obtain target training data; the speech-to-text module and the text-to-speech module are respectively determined as a feedback module and a labeled training module according to the data type of the target training data, and the feedback module is used to perform data conversion on the target training data to obtain feedback data, and the labeled training module is used to perform data conversion on the feedback data to obtain converted data, wherein the data types of the target training data and the converted data are the same, and the data types of the target training data and the feedback data are different, the data type being text type or speech type; the difference between the target training data and the converted data is calculated using a pre-constructed loss function to obtain a target loss value; when the target loss value is not less than a loss threshold, the model parameters of the initial speech synthesis model are updated, and the step of arbitrarily selecting a training data set from the financial business knowledge training dataset is returned; when the target loss value is less than the loss threshold, the initial speech synthesis model at this time is output to obtain the target speech synthesis model; The speech synthesis module is used to synthesize speech by using the text-to-speech module in the target speech synthesis model when receiving text containing financial business knowledge to be explained and popularized, thereby obtaining synthesized speech.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory that is communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the text-based speech synthesis method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the text-based speech synthesis method as described in any one of claims 1 to 5.

Images