Voice decoupling method and device, electronic equipment and computer readable storage medium

Abstract

The application relates to the technical field of financial technology, and provides a voice decoupling method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: obtaining a to-be-decoupled voice signal; inputting the decoupled voice signal into a pre-trained encoder for coding processing to obtain a hidden state vector; performing quantization fusion processing on the hidden state vector to obtain acoustic features; performing vector coding processing on the hidden state vector to obtain a coding vector; performing mask processing on the coding vector to obtain a mask vector; inputting the mask vector into a pre-trained network model to obtain hidden state information; performing decoding processing on the acoustic features to obtain reconstructed acoustic information; and performing conversion processing on the hidden state information to obtain semantic identification information. Through the technical scheme, the acoustic information and the semantic information are decoupled, so that the accuracy of semantic feedback of an intelligent answering system is improved.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of financial technology, and particularly to a voice decoupling method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the continuous development of society and economy and the advancement of technology, people's living standards have greatly improved. In the financial industry, financial institutions are increasingly using intelligent dialogue systems to respond to user inquiries in order to reduce the workload of their staff. However, when users send voice inquiries to intelligent dialogue systems, if there is background noise such as music, baby talk, or animal noises, the intelligent dialogue system may not be able to accurately answer the user's questions, resulting in a poor user experience. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0004] To address the problems mentioned in the background section, embodiments of this application provide a voice decoupling method, apparatus, electronic device, and computer-readable storage medium, which decouples acoustic information from semantic information, thereby improving the accuracy of semantic feedback in intelligent response systems.

[0005] In a first aspect, embodiments of this application provide a voice decoupling method, including:

[0006] Acquire the speech signal to be decoupled;

[0007] The speech signal to be decoupled is input into a pre-trained encoder for encoding to obtain a hidden state vector;

[0008] The hidden state vector is quantized and fused to obtain acoustic features; and the hidden state vector is vector encoded to obtain an encoded vector.

[0009] The encoded vector is masked to obtain a mask vector, and the mask vector is input into a pre-trained network model to obtain hidden state information;

[0010] The acoustic features are decoded to obtain reconstructed acoustic information; and the hidden state information is transformed to obtain semantic identification information.

[0011] According to some embodiments of this application, the encoder includes a first convolutional module, a convolutional block module, a long short-term memory recurrent network model, and a second convolutional module. The step of inputting the speech signal to be decoupled into a pre-trained encoder for encoding processing to obtain a hidden state vector includes:

[0012] The first convolution module performs a first feature extraction on the speech signal to be decoupled to obtain initial speech features;

[0013] The initial speech features are vectorized according to the convolutional block module to obtain vectorized speech features;

[0014] The vectorized speech features are analyzed and processed according to the Long Short-Term Memory Recursive Network Model to obtain speech vectorization analysis information.

[0015] The hidden state vector is obtained by performing a second feature extraction on the speech vectorization analysis information based on the second convolution module.

[0016] According to some embodiments of this application, the step of quantizing and fusing the hidden state vector to obtain acoustic features includes:

[0017] The hidden state vector is subjected to parallel vector quantization to obtain multiple quantization codes;

[0018] The acoustic features are obtained by fusing multiple quantization codes.

[0019] According to some embodiments of this application, the network model includes a preprocessing module, an encoding module, and a decoding module. The step of inputting the mask vector into the pre-trained network model to obtain hidden state information includes:

[0020] The preprocessing module performs data preprocessing on the mask vector to obtain a preprocessed vector.

[0021] The encoding module performs feature extraction on the preprocessed vector to obtain vector feature information;

[0022] The hidden state information is obtained by extracting data from the vector feature information using the decoding module.

[0023] According to some embodiments of this application, the step of converting the hidden state information to obtain semantic identifier information includes:

[0024] The hidden state information is masked using a pre-trained mask layer to obtain mask semantic information.

[0025] The semantic identifier information is obtained by performing sequence transformation on the mask semantic information.

[0026] According to some embodiments of this application, after converting the hidden state information to obtain semantic identifier information, the method further includes:

[0027] The semantic identifier information is matched with a preset question database to obtain the matching result;

[0028] The answer result is determined based on the matching result and corresponds to the semantic identifier information.

[0029] According to some embodiments of this application, determining the answer result corresponding to the semantic identifier information based on the matching result includes:

[0030] If the matching result indicates that there is answer information in the question database that corresponds to the semantic identifier information, then the answer information is determined as the answer result;

[0031] If the matching result indicates that there is no answer information corresponding to the semantic identifier information in the question database, the preset notification information will be used as the answer result.

[0032] Secondly, embodiments of this application also provide a voice decoupling device, the device comprising:

[0033] The first processing module is used to acquire the speech signal to be decoupled;

[0034] The second processing module is used to input the speech signal to be decoupled into a pre-trained encoder for encoding processing to obtain a hidden state vector;

[0035] The third processing module is used to perform quantization and fusion processing on the hidden state vector to obtain acoustic features; and to perform vector encoding processing on the hidden state vector to obtain an encoded vector;

[0036] The fourth processing module is used to perform masking processing on the encoded vector to obtain a mask vector, and input the mask vector into a pre-trained network model to obtain hidden state information;

[0037] The fifth processing module is used to decode the acoustic features to obtain reconstructed acoustic information; and to convert the hidden state information to obtain semantic identification information.

[0038] Thirdly, embodiments of this application also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the voice decoupling method described in the first aspect above.

[0039] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for performing the voice decoupling method described in the first aspect above.

[0040] The speech decoupling method according to the embodiments provided in this application has at least the following beneficial effects: In the process of speech decoupling, firstly, the speech signal to be decoupled is acquired; then, the speech signal to be decoupled is input into a pre-trained encoder for encoding processing to obtain a hidden state vector; then, the hidden state vector is quantized and fused to obtain acoustic features; and the hidden state vector is vector-encoded to obtain an encoded vector; then, the encoded vector is masked to obtain a mask vector; then, the mask vector is input into a pre-trained network model to obtain hidden state information; finally, the acoustic features are decoded to obtain reconstructed acoustic information; and the hidden state information is transformed to obtain semantic identifier information. Through the above technical solution, the speech signal to be decoupled can be decoupled into reconstructed acoustic information and semantic identifier information, thereby decoupling the acoustic and semantic information and improving the accuracy of semantic feedback in the intelligent answering system. Attached Figure Description

[0041] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0042] Figure 1 This is a flowchart of a voice decoupling method provided in one embodiment of this application;

[0043] Figure 2 This is a detailed flowchart of step S200 provided in one embodiment of this application;

[0044] Figure 3 This is a detailed flowchart of step S300 provided in one embodiment of this application;

[0045] Figure 4 This is a detailed flowchart of step S400 provided in one embodiment of this application;

[0046] Figure 5 This is a detailed flowchart of step S500 provided in one embodiment of this application;

[0047] Figure 6 This is a detailed flowchart of step S600 provided in one embodiment of this application;

[0048] Figure 7 This is a detailed flowchart of step S620 provided in one embodiment of this application;

[0049] Figure 8 This is a schematic diagram of a speech recognition device provided in one embodiment of this application;

[0050] Figure 9This is a schematic diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0053] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0054] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0055] AI is a new technical science that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. Artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. Artificial intelligence can simulate the information processes of human consciousness and thought. Furthermore, artificial intelligence utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results—the theories, methods, technologies, and application systems available for use.

[0056] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0057] Artificial intelligence, or AI, is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0058] The servers involved in artificial intelligence technology can be standalone servers or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0059] This application provides a speech decoupling method, apparatus, electronic device, and computer-readable storage medium. In the speech decoupling process, the speech signal to be decoupled is first acquired; then, the speech signal to be decoupled is input into a pre-trained encoder for encoding processing to obtain a hidden state vector; next, the hidden state vector is quantized and fused to obtain acoustic features; and then the hidden state vector is vector-encoded to obtain an encoded vector; next, the encoded vector is masked to obtain a mask vector; then, the mask vector is input into a pre-trained network model to obtain hidden state information; finally, the acoustic features are decoded to obtain reconstructed acoustic information; and the hidden state information is transformed to obtain semantic identifier information. Through the above technical solution, the speech signal to be decoupled can be decoupled into reconstructed acoustic information and semantic identifier information, thereby improving the accuracy of semantic feedback in intelligent response systems.

[0060] The voice decoupling method provided in this application relates to the field of financial technology. The voice decoupling method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the voice decoupling method, but is not limited to the above forms.

[0061] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0062] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0063] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0064] like Figure 1 As shown, Figure 1This is a flowchart of a voice decoupling method provided in one embodiment of this application. The voice decoupling method includes, but is not limited to, steps S100 to S500.

[0065] Step S100: Obtain the voice signal to be decoupled;

[0066] Step S200: Input the speech signal to be decoupled into the pre-trained encoder for encoding processing to obtain the hidden state vector;

[0067] Step S300: Quantize and fuse the hidden state vector to obtain acoustic features; and perform vector encoding on the hidden state vector to obtain an encoded vector;

[0068] Step S400: Mask the encoded vector to obtain a mask vector, and input the mask vector into the pre-trained network model to obtain hidden state information;

[0069] Step S500: Decode the acoustic features to obtain reconstructed acoustic information; and transform the hidden state information to obtain semantic identification information.

[0070] It should be noted that in the process of speech decoupling, the speech signal to be decoupled is first acquired; then, the speech signal to be decoupled is input into a pre-trained encoder for encoding to obtain a hidden state vector; next, the hidden state vector is quantized and fused to obtain acoustic features; then, the hidden state vector is vector encoded to obtain an encoded vector; next, the encoded vector is masked to obtain a mask vector; then, the mask vector is input into a pre-trained network model to obtain hidden state information; finally, the acoustic features are decoded to obtain reconstructed acoustic information; and the hidden state information is transformed to obtain semantic identifier information. Through the above technical solution, the speech signal to be decoupled can be decoupled into reconstructed acoustic information and semantic identifier information, thereby improving the accuracy of semantic feedback.

[0071] It should be noted that the voice signal to be decoupled can be the voice signal generated when a user consults a question with an intelligent dialogue system in the financial industry. For example, when a user asks the intelligent dialogue system, "What does the current insurance policy cover?", but there is other noise in the environment, the intelligent dialogue system can collect and process the user's voice and the noise in the environment, and convert the collected voice into a voice signal to be decoupled. This voice signal can then be analyzed and decoupled to provide feedback to the user on their question. It is worth noting that in the process of obtaining the voice signal to be decoupled, this application embodiment always obtains the user's permission or consent first, and the collection, use, and processing of this data comply with relevant laws, regulations, and standards. Furthermore, when this application embodiment needs to obtain the user's sensitive personal information, it will obtain the user's separate permission or consent through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent will the necessary user-related data for the normal operation of this application embodiment be obtained. For example, before a user makes a voice inquiry to the intelligent dialogue system, the intelligent dialogue system can first ask the user in a pop-up window whether they agree to obtain the voice information. Only after the user answers "agree" by voice or clicks the "agree" button can the intelligent dialogue system continue with the subsequent work.

[0072] It is worth noting that the speech signal to be decoupled can be decomposed into linguistic information and acoustic information. Linguistic information can be compared to translated text, but it contains higher-level grammatical and semantic information, and the representation forms differ in different languages. For music, the expected extraction is lyrics, while for light music, the extraction is empty information. Acoustic information includes pitch, loudness, timbre, as well as speech rate, speaking / musical style, etc. For music / light music, the expected extraction is the main elements of accompaniment (melody, rhythm, meter, dynamics, register, timbre, harmony, counterpoint, mode, tonality, etc.), as well as the timbre contained in baby talk and animal sounds.

[0073] It is worth noting that in intelligent dialogue systems in the financial industry, for any audio recording collected from the intelligent dialogue system (such as human speech, singing with accompaniment, light music, baby talk, animal sounds, instrument sounds, and other sounds), traditional network models can only extract information from their respective domains. The coupling between these domain information cannot be explored, and there is no universal model to uniformly explore the decoupling modeling of language and acoustic information. Such decoupling modeling has a profound impact on the speech domain and cross-modal generation and classification tasks of speech and text. This application can effectively solve the problem of decoupling language and acoustic information, enabling accurate feedback processing based on the decoupled language information to answer user questions and provide a better user experience.

[0074] It is worth noting that by inputting the speech signal to be decoupled into a pre-trained encoder, feature extraction can be performed on the speech signal, thereby obtaining the hidden state vector. This prepares the groundwork for subsequent speech signal decoupling. It is important to note that the pre-trained encoder is a pre-trained encoder; based on this trained encoder, feature extraction can be performed on the speech signal to be decoupled, thus obtaining the hidden state vector.

[0075] It is worth noting that during acoustic decoupling, the corresponding acoustic features can be obtained by quantizing and fusing the hidden state vector; during semantic decoupling, the hidden state vector can be encoded to obtain the encoded vector; and the acoustic and semantic decoupling processes can be executed in parallel, which can greatly speed up the efficiency of speech decoupling.

[0076] It's important to note that masking the encoded vector involves concealing certain features, preventing them from affecting subsequent analysis and improving accuracy. Masking is typically used to handle partial information in inputs or outputs, such as padding locations, invalid data, or neurons that need to be randomly discarded, to prevent these elements from negatively impacting model training and inference. In natural language processing, masking is often used for variable-length input sequences, such as masking padding locations to avoid affecting model recognition.

[0077] It is worth noting that decoding acoustic features using a decoder yields reconstructed acoustic information. Here, the decoder and encoder are relative, making the decoupling of acoustic information simpler, faster, and more accurate. Transforming the hidden state information yields semantic identifier information, further accelerating semantic decoupling.

[0078] In some embodiments, such as Figure 2 As shown, the encoder includes a first convolutional module, a convolutional block module, a long short-term memory recurrent network model, and a second convolutional module. The above step S200 may include, but is not limited to, steps S210 to S240.

[0079] Step S210: Extract initial speech features from the first feature of the speech signal to be decoupled using the first convolution module;

[0080] Step S220: Vectorize the initial speech features by performing vectorization processing on the convolutional block module to obtain vectorized speech features;

[0081] Step S230: Analyze and process the vectorized speech features according to the Long Short-Term Memory Recurrent Network Model to obtain speech vectorization analysis information;

[0082] Step S240: The hidden state vector is obtained by performing second feature extraction on the speech vectorization analysis information based on the second convolution module.

[0083] It should be noted that in the process of inputting the speech signal to be decoupled into the pre-trained encoder for encoding to obtain the hidden state vector, the initial speech features are first obtained by performing first feature extraction on the speech signal to be decoupled using the first convolutional module; then, the initial speech features are vectorized by the convolutional block module to obtain vectorized speech features; next, the vectorized speech features are analyzed by the Long Short-Term Memory recurrent network model to obtain speech vectorized analysis information; finally, the hidden state vector is obtained by performing second feature extraction on the speech vectorized analysis information using the second convolutional module. Through the above method, the hidden state vector can be reliably extracted from the speech signal to be decoupled, thus preparing the groundwork for subsequent speech decoupling.

[0084] It is worth noting that the Long Short-Term Memory (LSTM) recurrent neural network model is a type of temporal recurrent neural network specifically designed to address the long-term dependency problem inherent in general recurrent neural networks. All recurrent neural networks exhibit a chain-like structure of repeating neural network modules. Based on the LSM recurrent neural network model in this embodiment, vectorized speech features can be analyzed to obtain speech vectorization analysis information. Then, the second feature extraction process, performed on the speech vectorization analysis information by the second convolution module, yields the hidden state vector. This approach allows for more accurate decoupling of subsequent acoustic and semantic information.

[0085] For example, in the financial industry, when a user consults an intelligent dialogue system, the intelligent dialogue system can collect the speech signal to be decoupled. Then, the first convolution module can be used to extract the first feature of the speech signal to be decoupled to obtain the initial speech features, which prepares the groundwork for subsequent acoustic and semantic decoupling.

[0086] In some embodiments, such as Figure 3 As shown, step S300 may also include, but is not limited to, steps S310 to S320.

[0087] Step S310: Perform parallel vector quantization on the hidden state vector to obtain multiple quantized codes;

[0088] Step S320: The acoustic features are obtained by fusing multiple quantization codes.

[0089] It should be noted that in the process of quantizing and fusing the hidden state vector, the hidden state vector is first subjected to parallel vector quantization to obtain multiple quantized codes; then, the multiple quantized codes are fused to obtain the corresponding acoustic features, which prepares for subsequent speech reconstruction.

[0090] It is worth noting that parallel vector quantization of the hidden state vector can significantly improve the efficiency of quantization, which in turn can greatly improve the efficiency of speech reconstruction. As a result, in the financial industry, users can receive rapid responses when consulting intelligent dialogue systems, providing them with a better user experience.

[0091] In some embodiments, such as Figure 4 As shown, the network model includes a preprocessing module, an encoding module, and a decoding module. The above step S400 may also include, but is not limited to, steps S410, S420, and S430.

[0092] Step S410: The mask vector is preprocessed according to the preprocessing module to obtain the preprocessed vector;

[0093] Step S420: Extract vector feature information from the preprocessed vector using the encoding module;

[0094] Step S430: The hidden state information is obtained by extracting data from the vector feature information using the decoding module.

[0095] It should be noted that in the process of inputting the mask vector into the pre-trained network model to determine the hidden state information, the mask vector is first pre-processed by the preprocessing module in the network model to obtain the preprocessed vector; then, the preprocessed vector is extracted by the encoding module in the network model to obtain the vector feature information; finally, the hidden state information is obtained by extracting the vector feature information by the decoding module in the network model, which prepares for the subsequent semantic decoupling process.

[0096] It is worth noting that using the preprocessing module to preprocess the mask vector prepares the groundwork for subsequent feature extraction. Next, the encoding module extracts features from the preprocessed vector, yielding vector feature information. Finally, the decoding module, corresponding to the encoding module, processes the vector feature information to obtain the corresponding hidden state information, making the extraction of hidden state information more accurate.

[0097] In some embodiments, such as Figure 5 As shown, step S500 may also include, but is not limited to, steps S510 and S520.

[0098] Step S510: Mask the hidden state information based on the pre-trained mask layer to obtain mask semantic information;

[0099] Step S520: Perform sequence transformation processing on the mask semantic information to obtain semantic identifier information.

[0100] It should be noted that in the process of transforming the hidden state information, the mask semantic information can be obtained by first masking the hidden state information based on the mask layer; the semantic identification information can be obtained by performing sequence transformation on the mask semantic information. Through the above technical solution, the decoupling processing of semantic information can be made more accurate and convenient.

[0101] It is worth noting that by using a mask layer, specific information in the hidden state information can be masked, which can effectively prevent this specific information from affecting subsequent conversions. This can significantly improve the accuracy of the conversion and reduce the amount of data processing, thereby improving the efficiency of the conversion.

[0102] In some embodiments, such as Figure 6 As shown, after performing the above step S500, steps S610 and S620 may also be performed.

[0103] Step S610: Match the semantic identifier information with the preset question database to obtain the matching result;

[0104] Step S620: Determine the answer result corresponding to the semantic identifier information based on the matching result.

[0105] It should be noted that after converting the hidden state information to obtain semantic identifier information, the semantic identifier information can be matched with a preset question database to obtain a matching result. Then, the matching result is determined as the answer result corresponding to the semantic identifier information, and then the question feedback is given to the user, which brings a good user experience.

[0106] For example, in the banking industry, when a user consults an intelligent dialogue system, such as asking "What is the current annual interest rate for deposits?", the intelligent dialogue system, upon confirming that the semantic identifier information is "What is the current annual interest rate for deposits?", can perform question matching processing from a pre-set question database to determine the question corresponding to the semantic identifier information. Since each question in the question database corresponds to a question answer, the answer corresponding to the matched question can be used as the feedback result to the user. Thus, the user can receive feedback from the intelligent response system, providing a good user experience.

[0107] In some embodiments, such as Figure 7 As shown, step S620 may also include, but is not limited to, steps S621 and S622.

[0108] Step S621: If there is answer information corresponding to the semantic identifier information in the matching result representation question database, the answer information is determined as the answer result;

[0109] Step S622: If there is no answer information corresponding to the semantic identifier information in the matching result representation question database, the preset notification information is used as the answer result.

[0110] It should be noted that in the process of determining the answer result corresponding to the semantic identification information based on the matching results, if there is answer information corresponding to the semantic identification information in the matching result representation question database, the answer information can be determined as the answer result; if there is no answer information corresponding to the semantic identification information in the matching result representation question database, the preset notification information can be used as the answer result. In this way, a good user experience is provided.

[0111] For example, in the insurance industry, when a user consults an intelligent dialogue system about "what conditions need to be met to purchase insurance," if a corresponding answer exists in the question database, the system can provide that answer to the user. If no corresponding answer exists in the question database, the system can provide a pre-set notification to the user, such as "Sorry, your question cannot be answered at the moment," informing the user that their question cannot be answered automatically and they need to seek human assistance, thus improving the user experience.

[0112] To more clearly illustrate the voice decoupling method of this application, a specific embodiment is described below.

[0113] For the input speech, the speech information is first reduced to a hidden state vector by a convolutional neural network encoder. The convolutional neural network encoder consists of four parts: Conv1D, Conv block, LSTM and Conv1D. The hidden state vector is input to the acoustic decoder and semantic decoder respectively to obtain different information.

[0114] Acoustic decoder: First, the hidden state vector output by the convolutional neural network encoder is subjected to parallel vector quantization to obtain the quantized encoding of the hidden state vector. Then, the encodings are fused to output the acoustic features. The acoustic features are input into the acoustic decoder to reconstruct the speech. The acoustic decoder is a mirror structure of the convolutional neural network encoder, consisting of four parts: Conv1D, LSTM, Conv block, and Conv1D.

[0115] Meanwhile, the semantic decoder first performs vector encoding on the hidden state vector, and then passes the encoded vector through a mask layer as a random mask. This output is input into the transformer component to output the hidden state information, and then passes through the mask again to obtain the mask semantic information. Finally, the semantic identifier information is output.

[0116] In addition, such as Figure 8 As shown, one embodiment of this application also provides a voice decoupling device 10, including:

[0117] The first processing module 100 is used to acquire the speech signal to be decoupled;

[0118] The second processing module 200 is used to input the speech signal to be decoupled into a pre-trained encoder for encoding processing to obtain a hidden state vector;

[0119] The third processing module 300 is used to perform quantization and fusion processing on the hidden state vector to obtain acoustic features; and to perform vector encoding processing on the hidden state vector to obtain an encoded vector;

[0120] The fourth processing module 400 is used to perform masking processing on the encoded vector to obtain a mask vector, and input the mask vector into the pre-trained network model to obtain hidden state information;

[0121] The fifth processing module 500 is used to decode acoustic features to obtain reconstructed acoustic information and to transform hidden state information to obtain semantic identification information.

[0122] It should be noted that in the process of speech decoupling, the speech signal to be decoupled is first acquired; then, the speech signal to be decoupled is input into a pre-trained encoder for encoding to obtain a hidden state vector; next, the hidden state vector is quantized and fused to obtain acoustic features; then, the hidden state vector is vector encoded to obtain an encoded vector; next, the encoded vector is masked to obtain a mask vector; then, the mask vector is input into a pre-trained network model to obtain hidden state information; finally, the acoustic features are decoded to obtain reconstructed acoustic information; and the hidden state information is transformed to obtain semantic identifier information. Through the above technical solution, the speech signal to be decoupled can be decoupled into reconstructed acoustic information and semantic identifier information, thereby improving the accuracy of semantic feedback.

[0123] It is worth noting that by inputting the speech signal to be decoupled into a pre-trained encoder, feature extraction can be performed on the speech signal, thereby obtaining the hidden state vector. This prepares the groundwork for subsequent speech signal decoupling. It is important to note that the pre-trained encoder is a pre-trained encoder; based on this trained encoder, feature extraction can be performed on the speech signal to be decoupled, thus obtaining the hidden state vector.

[0124] It is worth noting that during acoustic decoupling, the corresponding acoustic features can be obtained by quantizing and fusing the hidden state vector; during semantic decoupling, the hidden state vector can be encoded to obtain the encoded vector; and the acoustic and semantic decoupling processes can be executed in parallel, which can greatly speed up the efficiency of speech decoupling.

[0125] It's important to note that masking the encoded vector involves concealing certain features, preventing them from affecting subsequent analysis and improving accuracy. Masking is typically used to handle partial information in inputs or outputs, such as padding locations, invalid data, or neurons that need to be randomly discarded, to prevent these elements from negatively impacting model training and inference. In natural language processing, masking is often used for variable-length input sequences, such as masking padding locations to avoid affecting model recognition.

[0126] It is worth noting that decoding acoustic features using a decoder yields reconstructed acoustic information. Here, the decoder and encoder are relative, making the decoupling of acoustic information simpler, faster, and more accurate. Transforming the hidden state information yields semantic identifier information, further accelerating semantic decoupling.

[0127] The specific implementation of the voice decoupling device 10 is basically the same as the specific embodiment of the voice decoupling method described above, and will not be repeated here.

[0128] In addition, such as Figure 9 As shown, one embodiment of this application also provides an electronic device 700, which includes: a memory 720, a processor 710, and a computer program stored on the memory 720 and executable on the processor 710.

[0129] The processor 710 and memory 720 can be connected via a bus or other means.

[0130] The non-transient software program and instructions required to implement the voice decoupling method of the above embodiments are stored in the memory 720. When executed by the processor 710, the voice decoupling methods of the above embodiments are executed, for example, the methods described above are executed. Figure 1 Method steps S100 to S500 Figure 2 Method steps S210 to S240 in the text Figure 3 Method steps S310 to S320 in the text Figure 4 Method steps S410 to S430, Figure 5 Method steps S510 to S520 in the text Figure 6 Method steps S610 to S620 and Figure 7 Method steps S621 to S622.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Furthermore, one embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor 710 or a controller, for example, by a processor 710 in the above-described device embodiment, causing the processor 710 to perform the voice decoupling method in the above-described embodiment, for example, to perform the above-described... Figure 1 Method steps S100 to S500 Figure 2 Method steps S210 to S240 in the text Figure 3 Method steps S310 to S320 in the text Figure 4 Method steps S410 to S430, Figure 5 Method steps S510 to S520 in the text Figure 6 Method steps S610 to S620 and Figure 7 Method steps S621 to S622.

[0133] The above embodiments can be used in combination, and modules with the same name in different embodiments may be the same or different.

[0134] The foregoing has described specific embodiments of this application; other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0135] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and computer-readable storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0136] The apparatus, device, computer-readable storage medium and method provided in the embodiments of this application are corresponding. Therefore, the apparatus, device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, device and computer storage medium will not be described again here.

[0137] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0138] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0139] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0140] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing the embodiments of this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0141] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0143] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0145] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0146] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0147] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0148] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0149] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0150] The embodiments of this application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0151] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0152] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A voice decoupling method, characterized in that, include: Acquire the speech signal to be decoupled; The speech signal to be decoupled is input into a pre-trained encoder for encoding to obtain a hidden state vector; The acoustic features are obtained by quantizing and fusing the hidden state vectors. And the hidden state vector is vector encoded to obtain the encoded vector; The encoded vector is masked to obtain a mask vector, and the mask vector is input into a pre-trained network model to obtain hidden state information; The acoustic features are decoded to obtain reconstructed acoustic information; and the hidden state information is transformed to obtain semantic identification information.

2. The voice decoupling method according to claim 1, characterized in that, The encoder includes a first convolutional module, a convolutional block module, a long short-term memory recurrent network model, and a second convolutional module. The step of inputting the speech signal to be decoupled into the pre-trained encoder for encoding processing to obtain the hidden state vector includes: The first convolution module performs a first feature extraction on the speech signal to be decoupled to obtain initial speech features; The initial speech features are vectorized according to the convolutional block module to obtain vectorized speech features; The vectorized speech features are analyzed and processed according to the Long Short-Term Memory Recursive Network Model to obtain speech vectorization analysis information. The hidden state vector is obtained by performing a second feature extraction on the speech vectorization analysis information based on the second convolution module.

3. The voice decoupling method according to claim 1, characterized in that, The process of quantizing and fusing the hidden state vector to obtain acoustic features includes: The hidden state vector is subjected to parallel vector quantization to obtain multiple quantization codes; The acoustic features are obtained by fusing multiple quantization codes.

4. The voice decoupling method according to claim 1, characterized in that, The network model includes a preprocessing module, an encoding module, and a decoding module. The step of inputting the mask vector into the pre-trained network model to obtain hidden state information includes: The preprocessing module performs data preprocessing on the mask vector to obtain a preprocessed vector. The encoding module performs feature extraction on the preprocessed vector to obtain vector feature information; The hidden state information is obtained by extracting data from the vector feature information using the decoding module.

5. The voice decoupling method according to claim 1, characterized in that, The process of transforming the hidden state information to obtain semantic identifier information includes: The hidden state information is masked using a pre-trained mask layer to obtain mask semantic information. The semantic identifier information is obtained by performing sequence transformation on the mask semantic information.

6. The voice decoupling method according to claim 1, characterized in that, After converting the hidden state information to obtain semantic identifier information, the method further includes: The semantic identifier information is matched with a preset question database to obtain the matching result; The answer result is determined based on the matching result and corresponds to the semantic identifier information.

7. The voice decoupling method according to claim 6, characterized in that, The step of determining the answer result corresponding to the semantic identifier information based on the matching result includes: If the matching result indicates that there is answer information in the question database that corresponds to the semantic identifier information, then the answer information is determined as the answer result; If the matching result indicates that there is no answer information corresponding to the semantic identifier information in the question database, the preset notification information will be used as the answer result.

8. A voice decoupling device, characterized in that, include: The first processing module is used to acquire the speech signal to be decoupled; The second processing module is used to input the speech signal to be decoupled into a pre-trained encoder for encoding processing to obtain a hidden state vector; The third processing module is used to perform quantization and fusion processing on the hidden state vector to obtain acoustic features; And the hidden state vector is vector encoded to obtain the encoded vector; The fourth processing module is used to perform masking processing on the encoded vector to obtain a mask vector, and input the mask vector into a pre-trained network model to obtain hidden state information; The fifth processing module is used to decode the acoustic features to obtain reconstructed acoustic information; and to convert the hidden state information to obtain semantic identification information.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the voice decoupling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the speech decoupling method according to any one of claims 1 to 7.

Images