Voice conversion method, device, apparatus and storage medium

Abstract

The present application relates to artificial intelligence technology, disclose a kind of voice conversion method, comprising: arbitrarily selecting one training data in training data set, obtain target training data;Respectively extract the voice feature of training voice and conversion speech label in target training data;Using the voice feature of the extracted training voice in conversion model is converted, and conversion voice feature is obtained;Conversion model is judged whether to converge based on the difference between conversion voice feature and the voice feature of extracted conversion speech label;When conversion model does not converge, the model parameters of conversion model are updated by second-order gradient self-adapting adjustment, and return arbitrarily selecting one training data step;When conversion model converges, the voice conversion of the voice to be converted is carried out using the conversion model at this time, and target conversion voice is obtained.The present application also proposes a kind of voice conversion device, equipment and medium, which can be used in financial field, and improves the efficiency of voice conversion of insurance notice matters for attention explanation voice.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence technology and financial technology, and in particular to a voice conversion method, device, electronic device and storage medium. Background Technology

[0002] In the financial services sector, to enhance users' understanding of relevant financial services, voice explanations are used to explain related financial knowledge. For example, in insurance, to help users better understand the precautions for insurance application, voice explanations are used to explain these precautions. At the same time, to improve user acceptance, the explanations can be converted into a voice type that customers prefer. Therefore, voice conversion is necessary.

[0003] Existing speech conversion methods require training a speech conversion model to perform speech conversion. However, the training process of existing models uses first-order gradient parameter iterative updates, which results in slow model convergence and long training time, leading to low efficiency in speech conversion of insurance policy explanations. Summary of the Invention

[0004] This invention provides a voice conversion method, device, electronic device, and storage medium, the main purpose of which is to improve the efficiency of voice conversion for explaining insurance policy application precautions.

[0005] Obtain a financial business knowledge training dataset and an initial speech conversion model, wherein each training data in the financial business knowledge training dataset includes: training speech and the corresponding converted speech label;

[0006] Arbitrarily select one training data point from the financial business knowledge training dataset to obtain the target training data;

[0007] Extract the speech features from the training speech in the target training data to obtain the training speech features;

[0008] The training speech features are converted using the initial speech conversion model to obtain converted speech features.

[0009] Extract the speech features of the converted speech labels from the target training data to obtain the label speech features;

[0010] The difference between the converted speech features and the labeled speech features is calculated using a pre-constructed loss function to obtain the target loss value;

[0011] Based on the target loss value, determine whether the initial speech conversion model has converged;

[0012] If the initial speech conversion model does not converge, the model parameters of the initial speech conversion model are adaptively adjusted and updated using the second-order gradient, and the process of arbitrarily selecting one training data from the financial business knowledge training dataset is returned.

[0013] When the initial speech conversion model converges, the initial speech conversion model at this time is output to obtain the target speech conversion model;

[0014] When receiving the speech containing financial business knowledge to be converted, the speech is converted using the target speech conversion model to obtain the target converted speech.

[0015] Optionally, the step of extracting speech features from the target training data to obtain training speech features includes:

[0016] The training speech is pre-emphasized to obtain standard training speech;

[0017] The standard training speech is segmented and windowed to obtain multiple speech frames;

[0018] Perform a discrete Fourier transform on each of the speech frames to convert the time-domain features of the speech frames into frequency-domain features, thereby obtaining the amplitude spectrum of each speech frame.

[0019] The amplitude spectrum of the speech frame is filtered, and the logarithm of the filtered amplitude spectrum is taken to obtain the standard amplitude spectrum of the speech frame.

[0020] The speech frame feature vector is obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame.

[0021] The speech frame feature vectors of all the speech frames are combined to obtain the training speech features.

[0022] Optionally, performing a discrete cosine transform on the standard amplitude spectrum of the speech frame to obtain the speech frame feature vector includes:

[0023] The static speech feature parameters of the speech frame are obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame.

[0024] The speech frame feature vector is obtained by concatenating all parameter values ​​in the static speech parameters of the speech frame in sequence into a vector form.

[0025] Optionally, performing a discrete cosine transform on the standard amplitude spectrum of the speech frame to obtain the speech frame feature vector includes:

[0026] The static speech feature parameters of the speech frame are obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame.

[0027] The static speech feature parameters of the speech frame are differentially processed to obtain the dynamic speech feature parameters of the speech frame.

[0028] The static and dynamic speech parameters of the speech frame are concatenated to obtain the speech frame feature vector of the speech frame.

[0029] Optionally, combining the speech frame feature vectors of all the speech frames to obtain the training speech features includes:

[0030] The feature vectors of each speech frame are combined according to the order of the standard training speech in the corresponding speech frame to obtain a speech feature vector sequence;

[0031] The training speech features are obtained by replacing each row in the speech feature vector sequence with the average value of all elements in that row.

[0032] Optionally, the step of using the target speech conversion model to convert the speech of the financial business knowledge to be converted into the target converted speech includes:

[0033] Extract the speech features of the financial business knowledge speech to be converted to obtain the speech features to be converted;

[0034] The speech features to be converted are input into the target speech conversion model to perform speech feature conversion, thereby obtaining the target speech features;

[0035] The target speech features are mapped to speech to obtain the target converted speech.

[0036] To address the above problems, the present invention also provides a speech conversion device, the device comprising:

[0037] The data acquisition module is used to acquire a financial business knowledge training dataset and an initial speech conversion model. Each training data in the financial business knowledge training dataset includes: training speech and a corresponding converted speech label.

[0038] The model training module is used to: arbitrarily select one training data point from the financial business knowledge training dataset to obtain target training data; extract speech features from the training speech in the target training data to obtain training speech features; use the initial speech conversion model to convert the training speech features to obtain converted speech features; extract speech features from the converted speech labels in the target training data to obtain label speech features; calculate the difference between the converted speech features and the label speech features using a pre-constructed loss function to obtain a target loss value; determine whether the initial speech conversion model has converged based on the target loss value; if the initial speech conversion model has not converged, perform second-order gradient adaptive adjustment and update on the model parameters of the initial speech conversion model, and return to the step of arbitrarily selecting one training data point from the financial business knowledge training dataset.

[0039] When the initial speech conversion model converges, the initial speech conversion model at this time is output to obtain the target speech conversion model;

[0040] The speech conversion module is used to receive financial business knowledge speech to be converted, and then use the target speech conversion model to convert the financial business knowledge speech to be converted into the target converted speech.

[0041] Optionally, the step of extracting speech features from the target training data to obtain training speech features includes:

[0042] The training speech is pre-emphasized to obtain standard training speech;

[0043] The standard training speech is segmented and windowed to obtain multiple speech frames;

[0044] Perform a discrete Fourier transform on each of the speech frames to convert the time-domain features of the speech frames into frequency-domain features, thereby obtaining the amplitude spectrum of each speech frame.

[0045] The amplitude spectrum of the speech frame is filtered, and the logarithm of the filtered amplitude spectrum is taken to obtain the standard amplitude spectrum of the speech frame.

[0046] The speech frame feature vector is obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame.

[0047] The speech frame feature vectors of all the speech frames are combined to obtain the training speech features.

[0048] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0049] Memory, storing at least one computer program; and

[0050] The processor executes the computer program stored in the memory to implement the speech conversion method described above.

[0051] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the aforementioned speech conversion method.

[0052] In this embodiment of the invention, the model parameters of the initial speech conversion model are adaptively adjusted and updated using second-order gradients. When training the initial speech conversion model, the model parameters are updated using second-order gradients adaptively. Compared with iteratively updating the model parameters using first-order gradients, the training speed of the model is faster, thereby improving the efficiency of speech synthesis. Therefore, the speech conversion method, device, electronic device and readable storage medium proposed in this embodiment of the invention improve the efficiency of speech conversion for insurance policy application instructions. Attached Figure Description

[0053] Figure 1 This is a schematic flowchart of a speech conversion method provided in an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of a speech conversion device provided in an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the internal structure of an electronic device that implements a speech conversion method according to an embodiment of the present invention.

[0056] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0058] This invention provides a voice conversion method. The execution entity of the voice conversion method includes, but is not limited to, at least one of electronic devices, such as a server or a terminal, that can be configured to execute the method provided in this application. In other words, the voice conversion method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0059] Reference Figure 1 The flowchart shown is a speech conversion method provided in an embodiment of the present invention. In this embodiment, the speech conversion method includes the following steps:

[0060] S1. Obtain a financial business knowledge training dataset and an initial speech conversion model, wherein each training data in the financial business knowledge training dataset includes: training speech and a corresponding converted speech label;

[0061] In this embodiment of the invention, the financial business knowledge training dataset is a collection of training data, the training voice is an insurance application precaution explanation voice of a preset first voice type, such as an explanation voice of critical illness insurance application precautions, and the converted voice tag is a voice of a preset second voice type that is the same as the training voice content but different from the voice type. In this embodiment of the invention, there are no restrictions on the first voice type and the second voice type.

[0062] For example, the training speech in the training data A is the first speech type of the text reading of the insurance application precautions explanation text A, and the converted speech label in the training data A is the second speech type of the text reading of the insurance application precautions explanation text A.

[0063] Furthermore, in this embodiment of the invention, the initial speech conversion model is a deep learning network that can convert speech features after training. Optionally, the initial speech conversion model can be a deep learning network such as a recurrent neural network, a long short-term memory network, or a gated recurrent unit.

[0064] S2. Randomly select one training data from the financial business knowledge training dataset to obtain the target training data;

[0065] In this embodiment of the invention, one training data point is arbitrarily selected from the financial business knowledge training dataset to obtain the target training data.

[0066] S3. Extract the speech features of the training speech in the target training data to obtain the training speech features;

[0067] In this embodiment of the invention, in order to perform speech feature conversion on the training speech in the target training data, it is necessary to first extract the speech features of the training speech. Therefore, speech features of a preset type are extracted from the training speech in the target training data to obtain the training speech features.

[0068] Furthermore, the preset types mentioned in the embodiments of the present invention include, but are not limited to, speech feature types such as Mel frequency cepstral coefficients and linear prediction cepstral coefficients.

[0069] Specifically, in this embodiment of the invention, extracting speech features of a preset type from the target training data to obtain training speech features includes:

[0070] The training speech is pre-emphasized to obtain standard training speech;

[0071] The standard training speech is segmented and windowed to obtain multiple speech frames;

[0072] Perform a discrete Fourier transform on each of the speech frames to convert the time-domain features of the speech frames into frequency-domain features, thereby obtaining the amplitude spectrum of each speech frame.

[0073] The amplitude spectrum of the speech frame is filtered, and the logarithm of the filtered amplitude spectrum is taken to obtain the standard amplitude spectrum of the speech frame.

[0074] The speech frame feature vector is obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame.

[0075] The speech frame feature vectors of all the speech frames are combined to obtain the training speech features.

[0076] Specifically, in this embodiment of the invention, performing a discrete cosine transform on the standard amplitude spectrum of the speech frame to obtain the speech frame feature vector includes:

[0077] The static speech feature parameters of the speech frame are obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame.

[0078] The speech frame feature vector is obtained by concatenating all the values ​​of the static speech parameters of the speech frame in sequence into a vector form.

[0079] In another embodiment of the present invention, performing a discrete cosine transform on the standard amplitude spectrum of the speech frame to obtain the speech frame feature vector includes:

[0080] The static speech feature parameters of the speech frame are obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame.

[0081] The static speech feature parameters of the speech frame are differentially processed to obtain the dynamic speech feature parameters of the speech frame.

[0082] The static and dynamic speech parameters of the speech frame are concatenated to obtain the speech frame feature vector of the speech frame.

[0083] In detail, in this embodiment of the invention, the static speech parameters and the filter types corresponding to the filtering processing are corresponding. When the amplitude spectrum of the speech frame is filtered using a Mel filter bank, the dynamic speech feature parameters of the speech frame are the Mel cepstral coefficients of the speech frame; when the amplitude spectrum of the speech frame is filtered using a triangular filter bank, the dynamic speech feature parameters of the speech frame are the triangular cepstral coefficients of the speech frame.

[0084] Furthermore, in this embodiment of the invention, combining the speech frame feature vectors of all the speech frames to obtain the training speech features includes:

[0085] The feature vectors of each speech frame are combined according to the order of the standard training speech in the corresponding speech frame to obtain a speech feature vector sequence;

[0086] The training speech features are obtained by replacing each row in the speech feature vector sequence with the average value of all elements in that row.

[0087] S4. Use the initial speech conversion model to convert the training speech features to obtain converted speech features;

[0088] In this embodiment of the invention, the trained speech features are input into the initial speech conversion model to obtain the converted speech features.

[0089] S5. Extract the speech features of the converted speech labels from the target training data to obtain the label speech features;

[0090] In this embodiment of the invention, the method for extracting the speech features of the converted speech tags in the target training data is similar to that used in S3, and will not be repeated here.

[0091] S6. Calculate the difference between the converted speech features and the labeled speech features using a pre-constructed loss function to obtain the target loss value;

[0092] In this embodiment of the invention, in order to measure the training progress of the model, the difference between the converted speech features and the labeled speech features is calculated using a pre-constructed loss function to obtain the target loss value.

[0093] Optionally, the loss function described in the embodiments of the present invention includes, but is not limited to: squared loss function, absolute value loss function, and cross-entropy loss function.

[0094] S7. Determine whether the initial speech conversion model has converged based on the target loss value;

[0095] In this embodiment of the invention, in order to measure the training progress of the initial speech conversion model, it is determined whether the initial speech conversion model has converged based on the target loss value.

[0096] Determining whether a model has converged using the loss value is a common technique in this field, and the specific implementation steps will not be elaborated here.

[0097] S8. When the initial speech conversion model does not converge, the model parameters of the initial speech conversion model are adaptively adjusted and updated using the second-order gradient, and the step of arbitrarily selecting one training data from the financial business knowledge training dataset is returned.

[0098] In this embodiment of the invention, when the initial speech conversion model does not meet the preset iteration requirements, it indicates that the initial speech conversion model still needs to be trained. Therefore, the model parameters of the initial speech conversion model are updated, and the step of arbitrarily selecting one training data from the financial business knowledge training dataset is returned.

[0099] Furthermore, in this embodiment of the invention, in order to better update the model parameters of the initial speech conversion model and improve the efficiency of model training, the step of performing second-order gradient adaptive adjustment and updating of the model parameters of the initial speech conversion model includes:

[0100] Obtain the model parameter gradients corresponding to the model parameters of the initial speech conversion model;

[0101] Obtain the first precondition operator and the second precondition operator of the initial transformation model;

[0102] Calculate the outer product matrix of the gradients of the model parameters, and update the first conditional operator and the second preconditional operator respectively based on the outer product matrix;

[0103] The inverse square root matrices of the first precondition operator and the second precondition operator are calculated using Newton's algorithm to obtain the first update matrix and the second update matrix;

[0104] Obtain the learning rate of the initial speech conversion model, and update the model parameters using the model parameter gradient, the learning rate, the first update coefficient, and the second update coefficient.

[0105] Specifically, in the embodiments of the present invention, the first preconditioning operator and the second preconditioning operator of the initial transformation model are respectively equivalent to the L matrix and the R matrix in the shampoo algorithm, which will not be elaborated here.

[0106] Furthermore, in this embodiment of the invention, the model parameters are updated using the following formula:

[0107] H = -ηggb

[0108] Where H represents the updated model parameters, η represents the learning rate, a represents the first update matrix, b represents the second update matrix, and g represents the gradient of the model parameters.

[0109] In this embodiment of the invention, the model parameter gradient and the model parameters are both in matrix form.

[0110] S9. When the initial speech conversion model converges, output the initial speech conversion model at this time to obtain the target speech conversion model;

[0111] In this embodiment of the invention, when the initial speech conversion model converges, it indicates that the training of the model is complete, and the initial speech conversion model at this time is output to obtain the target speech conversion model.

[0112] S10. When receiving the financial business knowledge speech to be converted, the speech to be converted is converted using the target speech conversion model to obtain the target converted speech.

[0113] In this embodiment of the invention, the financial business knowledge speech to be converted is an insurance application notice speech to be converted, and the financial business knowledge speech to be converted is a speech of a preset first speech type.

[0114] Furthermore, in this embodiment of the invention, in order to convert the financial business knowledge speech to be converted into speech of a preset second speech type, the target speech conversion model is used to convert the financial business knowledge speech to be converted to obtain the target converted speech.

[0115] For example, the financial business knowledge speech to be converted is an insurance application notice speech to be converted. The preset first speech type is Mandarin speech, and the preset second speech type is Northeastern dialect speech. In order to convert the insurance application notice speech to be converted from Mandarin to Northeastern dialect, the target speech conversion model is used to convert the insurance application notice speech to be converted to obtain the target converted speech, which is the Northeastern dialect insurance application notice speech to be converted.

[0116] Furthermore, in this embodiment of the invention, the target speech conversion model is used to convert the speech of the financial business knowledge to be converted, resulting in the target converted speech, including:

[0117] Extract speech features of a preset type from the speech containing financial business knowledge to be converted to obtain the speech features to be converted;

[0118] The speech features to be converted are input into the target speech conversion model to perform speech feature conversion, thereby obtaining the target speech features;

[0119] The target speech features are mapped to speech to obtain the target converted speech.

[0120] In this embodiment of the invention, the method for extracting the speech features of the preset type of the financial business knowledge speech to be converted is similar to that used in S3, and will not be described again here.

[0121] Furthermore, in this embodiment of the invention, commonly used speech synthesis models in the art can be used to map the target speech features into speech to obtain the target converted speech.

[0122] Furthermore, in this embodiment of the invention, after using the target speech conversion model to convert the speech of the financial business knowledge to be converted into the target converted speech, the target converted speech is sent to a preset speech recording or playback device, such as a speaker or microphone.

[0123] like Figure 2 The diagram shown is a functional block diagram of the speech conversion device of the present invention.

[0124] The speech conversion device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the speech conversion device may include a data acquisition module 101, a model training module 102, and a speech conversion module 103. The module mentioned in the present invention may also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0125] In this embodiment, the functions of each module / unit are as follows:

[0126] The data acquisition module 101 is used to acquire a financial business knowledge training dataset and an initial speech conversion model. Each training data in the financial business knowledge training dataset includes: training speech and a corresponding converted speech label.

[0127] The model training module 102 is used to arbitrarily select one training data from the financial business knowledge training dataset to obtain target training data; extract speech features from the training speech in the target training data to obtain training speech features; use the initial speech conversion model to convert the training speech features to obtain converted speech features; extract speech features from the converted speech labels in the target training data to obtain label speech features; calculate the difference between the converted speech features and the label speech features using a pre-constructed loss function to obtain a target loss value; determine whether the initial speech conversion model has converged based on the target loss value; if the initial speech conversion model has not converged, perform second-order gradient adaptive adjustment and update on the model parameters of the initial speech conversion model, and return to the step of arbitrarily selecting one training data from the financial business knowledge training dataset; if the initial speech conversion model has converged, output the initial speech conversion model at this time to obtain the target speech conversion model.

[0128] The speech conversion module 103 is used to receive financial business knowledge speech to be converted, and then use the target speech conversion model to convert the financial business knowledge speech to be converted into the target converted speech.

[0129] In detail, each module in the speech conversion device 100 described in this embodiment of the invention employs the same methods as described above during use. Figure 1 The same technical means are used in the speech conversion method described in the article, and it can produce the same technical effect, so it will not be repeated here.

[0130] like Figure 3 The diagram shown is a structural schematic of the electronic device that implements the speech conversion method of the present invention.

[0131] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program, such as a voice conversion program, stored in the memory 11 and capable of running on the processor 10.

[0132] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as code for a voice conversion program, but also to temporarily store data that has been output or will be output.

[0133] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules (e.g., voice conversion programs) stored in the memory 11, and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0134] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The communication bus 12 is configured to enable communication between the memory 11 and at least one processor 10, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus.

[0135] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0136] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault classification circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0137] Optionally, the communication interface 13 may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device and other electronic devices.

[0138] Optionally, the communication interface 13 may further include a user interface, which may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0139] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0140] The speech conversion program stored in the memory 11 of the electronic device is a combination of multiple computer programs, which, when run in the processor 10, can achieve the following:

[0141] Obtain a financial business knowledge training dataset and an initial speech conversion model, wherein each training data in the financial business knowledge training dataset includes: training speech and the corresponding converted speech label;

[0142] Arbitrarily select one training data point from the financial business knowledge training dataset to obtain the target training data;

[0143] Extract the speech features from the training speech in the target training data to obtain the training speech features;

[0144] The training speech features are converted using the initial speech conversion model to obtain converted speech features.

[0145] Extract the speech features of the converted speech labels from the target training data to obtain the label speech features;

[0146] The difference between the converted speech features and the labeled speech features is calculated using a pre-constructed loss function to obtain the target loss value;

[0147] Based on the target loss value, determine whether the initial speech conversion model has converged;

[0148] If the initial speech conversion model does not converge, the model parameters of the initial speech conversion model are adaptively adjusted and updated using the second-order gradient, and the process of arbitrarily selecting one training data from the financial business knowledge training dataset is returned.

[0149] When the initial speech conversion model converges, the initial speech conversion model at this time is output to obtain the target speech conversion model;

[0150] When receiving the speech containing financial business knowledge to be converted, the speech is converted using the target speech conversion model to obtain the target converted speech.

[0151] Specifically, the processor 10's implementation method of the above-mentioned computer program can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0152] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or volatile. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0153] Embodiments of the present invention may also provide a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0154] Obtain a financial business knowledge training dataset and an initial speech conversion model, wherein each training data in the financial business knowledge training dataset includes: training speech and the corresponding converted speech label;

[0155] Arbitrarily select one training data point from the financial business knowledge training dataset to obtain the target training data;

[0156] Extract the speech features from the training speech in the target training data to obtain the training speech features;

[0157] The training speech features are converted using the initial speech conversion model to obtain converted speech features.

[0158] Extract the speech features of the converted speech labels from the target training data to obtain the label speech features;

[0159] The difference between the converted speech features and the labeled speech features is calculated using a pre-constructed loss function to obtain the target loss value;

[0160] Based on the target loss value, determine whether the initial speech conversion model has converged;

[0161] If the initial speech conversion model does not converge, the model parameters of the initial speech conversion model are adaptively adjusted and updated using the second-order gradient, and the process of arbitrarily selecting one training data from the financial business knowledge training dataset is returned.

[0162] When the initial speech conversion model converges, the initial speech conversion model at this time is output to obtain the target speech conversion model;

[0163] When receiving the speech containing financial business knowledge to be converted, the speech is converted using the target speech conversion model to obtain the target converted speech.

[0164] Furthermore, the computer's usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system, applications required for at least one function, etc.; and the data storage area may store data created based on the use of blockchain nodes, etc.

[0165] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0166] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0167] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0168] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0169] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0170] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0171] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0172] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A speech conversion method, characterized in that, The method includes: Obtain a financial business knowledge training dataset and an initial speech conversion model, wherein each training data in the financial business knowledge training dataset includes: training speech and the corresponding converted speech label; Arbitrarily select one training data point from the financial business knowledge training dataset to obtain the target training data; Extract the speech features from the training speech in the target training data to obtain the training speech features; The training speech features are converted using the initial speech conversion model to obtain converted speech features. Extract the speech features of the converted speech labels from the target training data to obtain the label speech features; The difference between the converted speech features and the labeled speech features is calculated using a pre-constructed loss function to obtain the target loss value; Based on the target loss value, determine whether the initial speech conversion model has converged; If the initial speech conversion model does not converge, the model parameters of the initial speech conversion model are adaptively adjusted and updated using the second-order gradient, and the process of arbitrarily selecting one training data from the financial business knowledge training dataset is returned. When the initial speech conversion model converges, the initial speech conversion model at this time is output to obtain the target speech conversion model; When the speech containing financial business knowledge to be converted is received, the speech is converted using the target speech conversion model to obtain the target converted speech.

2. The speech conversion method as described in claim 1, characterized in that, The step of extracting speech features from the target training data to obtain training speech features includes: The training speech is pre-emphasized to obtain standard training speech; The standard training speech is segmented and windowed to obtain multiple speech frames; Perform a discrete Fourier transform on each of the speech frames to convert the time-domain features of the speech frames into frequency-domain features, thereby obtaining the amplitude spectrum of each speech frame. The amplitude spectrum of the speech frame is filtered, and the logarithm of the filtered amplitude spectrum is taken to obtain the standard amplitude spectrum of the speech frame. The speech frame feature vector is obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame. The speech frame feature vectors of all the speech frames are combined to obtain the training speech features.

3. The speech conversion method as described in claim 2, characterized in that, The step of performing a discrete cosine transform on the standard amplitude spectrum of the speech frame to obtain the speech frame feature vector includes: The static speech feature parameters of the speech frame are obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame. The speech frame feature vector is obtained by concatenating all the parameter values ​​of the static speech feature parameters of the speech frame in order into a vector form.

4. The speech conversion method as described in claim 2, characterized in that, The step of performing a discrete cosine transform on the standard amplitude spectrum of the speech frame to obtain the speech frame feature vector includes: The static speech feature parameters of the speech frame are obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame. The static speech feature parameters of the speech frame are differentially processed to obtain the dynamic speech feature parameters of the speech frame. The static and dynamic speech feature parameters of the speech frame are concatenated to obtain the speech frame feature vector of the speech frame.

5. The speech conversion method as described in claim 2, characterized in that, The step of combining the speech frame feature vectors of all the speech frames to obtain the training speech features includes: The feature vectors of each speech frame are combined according to the order of the standard training speech in the corresponding speech frame to obtain a speech feature vector sequence; The training speech features are obtained by replacing each row in the speech feature vector sequence with the average value of all elements in that row.

6. The speech conversion method according to any one of claims 1 to 5, characterized in that, The step of using the target speech conversion model to convert the speech of the financial business knowledge to be converted, to obtain the target converted speech, includes: Extract the speech features of the financial business knowledge speech to be converted to obtain the speech features to be converted; The speech features to be converted are input into the target speech conversion model to perform speech feature conversion, thereby obtaining the target speech features; The target speech features are mapped to speech to obtain the target converted speech.

7. A voice conversion device, characterized in that, include: The data acquisition module is used to acquire a financial business knowledge training dataset and an initial speech conversion model. Each training data in the financial business knowledge training dataset includes: training speech and a corresponding converted speech label. The model training module is used to: arbitrarily select one training data point from the financial business knowledge training dataset to obtain target training data; extract speech features from the training speech in the target training data to obtain training speech features; use the initial speech conversion model to convert the training speech features to obtain converted speech features; extract speech features from the converted speech labels in the target training data to obtain label speech features; calculate the difference between the converted speech features and the label speech features using a pre-constructed loss function to obtain a target loss value; determine whether the initial speech conversion model has converged based on the target loss value; if the initial speech conversion model has not converged, perform second-order gradient adaptive adjustment and update on the model parameters of the initial speech conversion model, and return to the step of arbitrarily selecting one training data point from the financial business knowledge training dataset; if the initial speech conversion model has converged, output the current initial speech conversion model to obtain the target speech conversion model. The speech conversion module is used to convert the received financial business knowledge speech using the target speech conversion model to obtain the target converted speech.

8. The speech conversion device as described in claim 7, characterized in that, The step of extracting speech features from the target training data to obtain training speech features includes: The training speech is pre-emphasized to obtain standard training speech; The standard training speech is segmented and windowed to obtain multiple speech frames; Perform a discrete Fourier transform on each of the speech frames to convert the time-domain features of the speech frames into frequency-domain features, thereby obtaining the amplitude spectrum of each speech frame. The amplitude spectrum of the speech frame is filtered, and the logarithm of the filtered amplitude spectrum is taken to obtain the standard amplitude spectrum of the speech frame. The speech frame feature vector is obtained by performing a discrete cosine transform on the standard amplitude spectrum of the speech frame. The speech frame feature vectors of all the speech frames are combined to obtain the training speech features.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory that is communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the speech conversion method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the speech conversion method as described in any one of claims 1 to 6.

Images