Multitask voice detection method and device, storage medium and electronic equipment
By integrating each task model and generating a target voice detection model, the problem that the voice quality inspection method in the existing technology only supports single task optimization is solved, and more flexible and adaptable multi-task voice detection is achieved, improving voice quality The accuracy of the inspection.
Patent Information
- Application Number
- CN202411508955.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2025-02-07
AI Technical Summary
The existing voice quality inspection methods only support optimization for a single task, resulting in inaccurate voice quality inspection results and difficulty in adapting to changing quality inspection needs and changing business scenarios.
A multi-task speech detection method based on preset neural network model is adopted, and a target speech detection model is generated by fusing each task model to realize the detection of multiple tasks.
Improves the flexibility and adaptability of speech detection, ensuring the performance improvement of the new model on multiple tasks without performance losses, and provides more accurate voice quality inspection results.
Smart Images
Figure CN119400180A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of speech quality inspection technology, applicable to the field of artificial intelligence technology and medical health, and in particular to a multi-task speech detection method, device, storage medium and electronic device. Background Art
[0002] With the development of artificial intelligence technology, speech detection methods can support functions such as auxiliary disease diagnosis, health management, and remote consultation. Speech quality inspection is a key link in evaluating and improving the quality of customer service, involving the analysis of customer service conversation recordings to identify problems and provide improvement suggestions. Traditional speech quality inspection relies on manual listening and evaluation, but this method is time-consuming and costly. In recent years, automated speech quality inspection systems have begun to adopt deep learning technology to automatically identify problems through speech recognition and natural language processing technology. However, these systems are usually optimized for a single task and are difficult to adapt to changing quality inspection needs and changing business scenarios. Summary of the invention
[0003] In view of this, the present invention provides a multi-task speech detection method, device, storage medium and electronic device, the main purpose of which is to solve the problem that the current speech quality inspection method only supports optimization of a single task and the speech quality inspection results are inaccurate.
[0004] To solve the above problems, the present application provides a multi-task speech detection method, comprising:
[0005] Based on each corpus sample, a preset neural network model is used to perform model training for each task respectively to obtain a task model corresponding to each task;
[0006] Based on the first model parameter vector of the preset neural network model and the second model parameter vector of each of the task models, a preset model fusion method is used to fuse each of the task models to obtain a target speech detection model;
[0007] Get the voice input text to be detected;
[0008] The target speech detection model is used to detect the speech input text to be detected, and speech detection results corresponding to each of the tasks are obtained.
[0009] Optionally, based on each corpus sample, a preset neural network model is used to perform model training for each task to obtain a task model corresponding to each task, specifically including:
[0010] Labeling each of the corpus samples based on the task category of each of the tasks to obtain label data corresponding to each of the tasks;
[0011] For the label data corresponding to the same task, a preset cross entropy loss function is used to perform model training on a preset neural network model to generate a task model corresponding to the same task, so as to obtain the task models corresponding to each of the tasks respectively;
[0012] The tasks include emotion recognition, topic detection and speech rate assessment.
[0013] The preset neural network model includes a word embedding layer, a position encoding layer, multiple multi-head self-attention layers, multiple residual summation network layers, a feedforward neural network layer and a fully connected layer.
[0014] Optionally, the first model parameter vector based on the preset neural network model and the second model parameter vector of each task model are fused by a preset model fusion method to obtain a target speech detection model, specifically including:
[0015] Performing calculations based on the first model parameter vector and each of the second model parameter vectors to obtain a first task vector corresponding to each task;
[0016] Pruning each of the first task vectors to obtain each second task vector;
[0017] filtering the model parameters in each of the second task vectors to obtain each third task vector;
[0018] Perform mean merging based on each of the third task vectors to obtain a target model parameter vector of the target speech detection model;
[0019] The preset neural network model is used to construct a model based on the target model parameter vector to obtain the target speech detection model.
[0020] Optionally, the performing calculation based on the first model parameter vector and each of the second model parameter vectors to obtain a first task vector corresponding to each task specifically includes:
[0021] Subtraction operations are performed on each of the second model parameter vectors and the first model parameter vectors to obtain first task vectors corresponding to each of the tasks.
[0022] Optionally, pruning each of the first task vectors to obtain each of the second task vectors specifically includes:
[0023] For the same first task vector, performing absolute value calculation processing on a plurality of model parameters corresponding to each model structure layer, to obtain initial parameter values corresponding to each model structure layer;
[0024] Screening each of the initial parameter values to obtain each first parameter value and each second parameter value;
[0025] Perform numerical substitution on each of the second parameter values to obtain a third parameter value corresponding to each of the second parameter values;
[0026] Data is reorganized based on each of the first parameter values and each of the third parameter values to obtain the second task vector corresponding to the same first task vector.
[0027] Optionally, filtering the model parameters in each of the second task vectors to obtain each of the third task vectors specifically includes:
[0028] Performing an addition operation on the model parameters of each of the second task vectors at the same position in the same model structure layer to obtain model parameters and values corresponding to different positions in the same model structure layer;
[0029] The model parameters of each of the second task vectors are screened based on the model parameters and values corresponding to the same position of the same model structure layer to obtain each fourth parameter value and each fifth parameter value;
[0030] Perform numerical substitution on each of the fifth parameter values to obtain a sixth parameter value corresponding to each of the fifth parameter values;
[0031] Data is reorganized based on each of the fourth parameter values and each of the sixth parameter values to obtain a third task vector corresponding to each of the second task vectors.
[0032] Optionally, performing mean merging based on each of the third task vectors to obtain a target model parameter vector of the target speech detection model specifically includes:
[0033] Performing average calculation processing on the model parameters of each of the third task vectors at the same position in the same model structure layer to obtain an average task vector;
[0034] An addition operation is performed based on the first model parameter vector of the preset neural network model and the average task vector to obtain a target model parameter vector of the target speech detection model.
[0035] To solve the above problems, the present application provides a multi-task speech detection device, comprising:
[0036] A model training module is used to perform model training for each task based on each corpus sample using a preset neural network model to obtain a task model corresponding to each task;
[0037] A fusion module, used to fuse each of the task models using a preset model fusion method based on the first model parameter vector of the preset neural network model and the second model parameter vector of each of the task models to obtain a target speech detection model;
[0038] An acquisition module is used to acquire the voice input text to be detected;
[0039] The detection module is used to detect the speech input text to be detected using the target speech detection model to obtain speech detection results corresponding to each of the tasks.
[0040] In order to solve the above-mentioned problem, the present application provides a storage medium, characterized in that the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned multi-task speech detection method are implemented.
[0041] To solve the above problems, the present application provides an electronic device, characterized in that it includes at least a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned multi-task speech detection method when executing the computer program on the memory.
[0042] Beneficial effects of the present application: The present application obtains a target speech detection model by merging various task models. The merged target speech detection model can detect multiple tasks, thereby improving the flexibility and adaptability of speech detection. Since the interference between parameters is considered during the merging process, the performance of the new model on multiple tasks is improved without performance loss.
[0043] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0045] Figure 1 This is a schematic diagram of an application environment of a multi-task speech detection method in an embodiment of the present application;
[0046] Figure 2 A schematic diagram of a multi-task speech detection method provided in an embodiment of the present application is shown;
[0047] Figure 3 A schematic flow chart of a multi-task speech detection method provided by another embodiment of the present application is shown;
[0048] Figure 4 A structural block diagram of a multi-task speech detection device provided in yet another embodiment of the present application is shown. DETAILED DESCRIPTION
[0049] Various aspects and features of the present application are described herein with reference to the accompanying drawings.
[0050] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.
[0051] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.
[0052] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.
[0053] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.
[0054] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.
[0055] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.
[0056] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.
[0057] The multi-task speech detection method provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, the client communicates with the server through a network. The server can receive the voice input text to be detected through the client, and based on each corpus sample, use a preset neural network model to perform model training for each task respectively, and obtain a task model corresponding to each of the tasks; based on the first model parameter vector of the preset neural network model and the second model parameter vector of each of the task models, use a preset model fusion method to fuse each of the task models to obtain a target voice detection model; obtain the voice input text to be detected; use the target voice detection model to detect the voice input text to be detected, and obtain the voice detection results corresponding to each of the tasks, and feed the voice detection results back to the client. The present application can merge multiple single-task models to form a powerful multi-task target voice detection model. The merged model can handle a variety of quality inspection tasks, which improves the flexibility and adaptability of the system. Since the interference between parameters is considered during the merging process, the performance of the new model on multiple tasks has been improved without performance loss.
[0058] The present application embodiment provides a multi-task voice detection method, such as Figure 2 As shown, including:
[0059] Step S101: Based on each corpus sample, a preset neural network model is used to perform model training for each task to obtain a task model corresponding to each task;
[0060] In the specific implementation process of this step, the multi-task speech detection method of the present application can be applied to the medical cloud platform. Medical cloud refers to the use of "cloud computing" to create a medical and health service cloud platform based on new technologies such as cloud computing, mobile technology, multimedia, 4G communication, big data, and the Internet of Things, combined with medical technology, to achieve the sharing of medical resources and the expansion of medical scope. Because of the combination of cloud computing technology, the medical cloud improves the efficiency of medical institutions and facilitates residents to seek medical treatment. The medical cloud also has the advantages of data security, information sharing, dynamic expansion, and overall layout. Based on each of the corpus samples, each of the tasks is marked separately to obtain label data corresponding to each of the tasks; wherein each of the tasks includes emotion recognition tasks, topic detection tasks, and speech speed evaluation tasks; the present application detects the speech input text to be detected from multiple task types such as emotion recognition tasks, topic detection tasks, and speech speed evaluation tasks. Detecting the speech input text to be detected from multiple aspects can more effectively monitor and improve service quality, while reducing dependence on manual quality inspection. The preset neural network model is trained using a preset cross entropy loss function for the label data corresponding to the same task, and a task model corresponding to the same task is generated to obtain the task models corresponding to each task; the physical structure layer of the model structure of each task model is the same, but the model parameters are different. The preset neural network model includes a word embedding layer, a position encoding layer, multiple multi-head self-attention layers, multiple residual summation network layers, a feedforward neural network layer, and a fully connected layer.
[0061] Step S102: Based on the first model parameter vector of the preset neural network model and the second model parameter vector of each task model, a preset model fusion method is used to fuse each task model to obtain a target speech detection model;
[0062] In the specific implementation process of this step, calculation processing is performed based on the first model parameter vector and each of the second model parameter vectors to obtain the first task vectors corresponding to each task; each of the first task vectors is pruned to obtain each of the second task vectors; each of the first task vectors is pruned to retain a predetermined percentage of important model parameter values in each of the first task vectors to obtain each of the second task vectors. The model parameters in each of the second task vectors are filtered to obtain each of the third task vectors; the model parameters at the same position of the model structure layer are calculated and summed, and then the model parameters in the second task vectors are filtered based on the sum value obtained after the sum calculation, and the model parameters with the same positive or negative value as the sum value obtained after the sum calculation are retained, and the numbers with different positive or negative values from the sum value obtained after the sum calculation are replaced with target characters, and the target characters can be the number 0, and the target character values can be set according to actual needs. Mean merging is performed based on each of the third task vectors to obtain the target model parameter vector of the target speech detection model; the preset neural network model is used to construct the model based on the target model parameter vector to obtain the target speech detection model.
[0063] Step S103: Acquire the voice input text to be detected;
[0064] In the specific implementation process of this step, the user's voice data is received, and the voice data is recognized using a preset voice recognition method to obtain initial text data; the initial voice text data is preprocessed; and the voice input text to be detected is obtained. Specifically, the initial voice text data is cleaned to obtain cleaned data; the cleaned data is subjected to noise removal processing to obtain denoised data; the denoised data is subjected to punctuation removal processing to obtain cleaned voice data; and text is generated based on the cleaned voice data, the preset task prompt sentences corresponding to each of the tasks, and the task label category prompt sentences corresponding to each of the tasks, to obtain the voice input text to be detected.
[0065] Step S104: using the target speech detection model to detect the speech input text to be detected, and obtaining speech detection results corresponding to each of the tasks.
[0066] During the specific implementation of this step, the target speech detection model is used to detect the speech input text to be detected, and speech detection results corresponding to each of the tasks are obtained; specifically, the speech input text to be detected is input into the merged target speech detection model, and the target speech detection model outputs quality inspection results of multiple tasks, and the quality inspection results can be emotion scores, topic classifications, speech speed assessments and other results.
[0067] The present application obtains a target speech detection model by merging each task model. The merged target speech detection model can detect multiple tasks, thereby improving the flexibility and adaptability of speech detection. Since the interference between parameters is considered during the merging process, the performance of the new model on multiple tasks is improved without performance loss.
[0068] Another embodiment of the present application provides another multi-task speech detection method, such as Figure 3 As shown, including:
[0069] Step S201: labeling each of the corpus samples based on the task category of each task to obtain label data corresponding to each of the tasks;
[0070] In the specific implementation of this step, the task categories include emotion recognition tasks, topic detection tasks and speaking speed assessment tasks; when the task category is marked as emotion recognition tasks, the label data includes: emotion labels such as happy, sad, and angry; when the task category is marked as topic detection tasks, the label data includes: intention labels such as purchase, consultation, and complaint; when the task category is marked as speaking speed assessment, the label data includes: speaking speed labels such as fast speaking speed, slow speaking speed, and moderate speaking speed.
[0071] Step S202: for the label data corresponding to the same task, a preset cross entropy loss function is used to perform model training on a preset neural network model, and a task model corresponding to the same task is generated, so as to obtain the task models corresponding to each task respectively;
[0072] In the specific implementation process of this step, for the target task, the word embedding layer of the preset neural network model is used to process each of the label data to obtain an initial feature vector corresponding to each of the label data; the position encoding of each of the initial feature vectors is performed to obtain each first feature vector; the first multi-head self-attention layer of the preset neural network model is used to extract features of each of the first feature vectors to obtain each second feature vector; the first multi-head self-attention layer can capture the internal dependencies of the text sequence and help understand the interaction between each word in the first feature vector; based on the first residual network layer, each of the second feature vectors is normalized to obtain a third feature vector; the second multi-head self-attention layer of the preset neural network model is used to extract features of the third feature vector to obtain a fourth feature vector; based on the second residual network layer, each of the fourth feature vectors is normalized to obtain a fifth feature vector. vector; based on the feedforward neural network layer of the preset neural network model, the fifth eigenvector is subjected to nonlinear transformation processing to obtain the sixth eigenvector; based on the third residual network layer, each of the sixth eigenvectors is subjected to normalization processing to obtain each target eigenvector; based on each of the target eigenvectors, the transmission layer of the preset neural network model is used to perform calculation processing to obtain the prediction data corresponding to each of the label data; based on each of the label data, the prediction data corresponding to each of the label data, the model parameters of the preset neural network model are updated using the preset cross entropy loss function to generate the current neural network model; the current neural network model is trained by the preset cross entropy loss function using each of the label data until the preset conditions are met, and the task model corresponding to the target task is obtained by training; the preset conditions may be reaching the preset number of iterations, or the model accuracy meets the preset conditions, etc. To obtain the task models corresponding to each of the tasks. The preset sentiment analysis model can be applied to a variety of different fields, such as speech recognition, medical diagnosis, application testing, etc.
[0073] Step S203: performing calculation processing based on the first model parameter vector and each of the second model parameter vectors to obtain a first task vector corresponding to each of the tasks;
[0074] In the specific implementation process of this step, subtraction operations are performed on each of the second model parameter vectors and the first model parameter vectors to obtain first task vectors corresponding to each of the tasks. Specifically, based on the model structure hierarchy of the preset neural network model and each of the task models, a mapping relationship between each first model parameter in the first model parameter vector and each second model parameter in the second model parameter vector is constructed to obtain each mapping relationship; subtraction operations are performed on the second model parameters and the first model parameters corresponding to the same mapping relationship to obtain model parameter differences corresponding to each of the mapping relationships, so as to obtain first task vectors corresponding to each of the tasks.
[0075] Step S204: pruning each of the first task vectors to obtain each of the second task vectors;
[0076] In the specific implementation process of this step, for the same first task vector, the absolute value calculation processing is performed on the multiple model parameters corresponding to each model structure layer to obtain the initial parameter values corresponding to each model structure layer; each initial parameter value is screened to obtain each first parameter value and each second parameter value; each first parameter value is a preset number of initial parameter values with absolute values sorted first, and each initial parameter value with absolute values sorted later is each second parameter value. Each second parameter value is numerically replaced to obtain a third parameter value corresponding to each second parameter value; specifically, the third parameter value can be the value 0, and each second parameter value is replaced with the number 0. Data is reorganized based on each first parameter value and each third parameter value to obtain the second task vector corresponding to the same first task vector.
[0077] Step S205: filtering the model parameters in each of the second task vectors to obtain each third task vector;
[0078] In the specific implementation process of this step, the model parameters of each second task vector at the same position of the same model structure layer are added to obtain the model parameters and values corresponding to the different positions of the same model structure layer; the model parameters of each second task vector are screened based on the model parameters and values corresponding to the same position of the same model structure layer to obtain the fourth parameter values and the fifth parameter values; each fourth parameter value is a model parameter with the same trend as the model parameter and value of the second task vector; the same trend is a number that is positive, negative or 0; each fifth parameter value is a model parameter value with the opposite trend as the model parameter and value of the second task vector. Each fifth parameter value is numerically replaced to obtain the sixth parameter value corresponding to each fifth parameter value; each fifth parameter value is numerically replaced with a target character, and the target character can be the number 0. Data is reorganized based on each fourth parameter value and each sixth parameter value to obtain the third task vector corresponding to each second task vector.
[0079] Step S206: performing mean merging based on each of the third task vectors to obtain a target model parameter vector of the target speech detection model;
[0080] During the specific implementation of this step, the model parameters of each of the third task vectors at the same position in the same model structure layer are averaged to obtain an average task vector; and the first model parameter vector of the preset neural network model and the average task vector are added to obtain a target model parameter vector of the target speech detection model.
[0081] Step S207: constructing a model using the preset neural network model based on the target model parameter vector to obtain the target speech detection model;
[0082] During the specific implementation of this step, the preset neural network model is used to construct a model based on the target model parameter vector to obtain the target speech detection model; the preset neural network model, each of the task models and the target speech detection model of the present application have the same physical structure level but different model parameters.
[0083] Step S208: Acquire the voice input text to be detected;
[0084] In the specific implementation process of this step, the user's voice data is received, and the voice data is recognized using a preset voice recognition method to obtain initial text data; the initial voice text data is preprocessed; and the voice input text to be detected is obtained. Specifically, the initial voice text data is cleaned to obtain cleaned data; the cleaned data is subjected to noise removal processing to obtain denoised data; the denoised data is subjected to punctuation removal processing to obtain cleaned voice data; and text is generated based on the cleaned voice data, the preset task prompt sentences corresponding to each of the tasks, and the task label category prompt sentences corresponding to each of the tasks, to obtain the voice input text to be detected.
[0085] Step S209: using the target speech detection model to detect the speech input text to be detected, and obtaining speech detection results corresponding to each of the tasks.
[0086] During the specific implementation of this step, the target speech detection model is used to detect the speech input text to be detected, and speech detection results corresponding to each of the tasks are obtained; specifically, the speech input text to be detected is input into the merged target speech detection model, and the target speech detection model outputs quality inspection results of multiple tasks, and the quality inspection results can be emotion scores, topic classifications, speech speed assessments and other results.
[0087] This application adopts the preset neural network model as the basic model, and uses different corpora to train different task types to obtain task models corresponding to different task types; then the task models are merged into a comprehensive target speech detection model by pruning and merging methods, and finally the target speech detection model can be used to detect the speech input text to be detected to obtain the detection results corresponding to each task, thereby improving the flexibility and adaptability of the speech detection method. Since the interference between parameters is considered in the merging process, the performance of the target speech detection model on multiple tasks has been improved without performance loss, making the results of speech quality inspection more accurate.
[0088] Another embodiment of the present application provides a multi-task speech detection device, such as Figure 4 As shown, including:
[0089] Model training module 1, used to perform model training for each task based on each corpus sample using a preset neural network model to obtain a task model corresponding to each task;
[0090] A fusion module 2 is used to fuse each of the task models using a preset model fusion method based on the first model parameter vector of the preset neural network model and the second model parameter vector of each of the task models to obtain a target speech detection model;
[0091] Acquisition module 3, used to acquire the voice input text to be detected;
[0092] The detection module 4 is used to detect the speech input text to be detected using the target speech detection model to obtain speech detection results corresponding to each of the tasks.
[0093] During the specific implementation process, the model training module 1 is specifically used to: label each of the corpus samples based on the task category of each of the tasks, and obtain label data corresponding to each of the tasks; use a preset cross entropy loss function to perform model training on a preset neural network model for the label data corresponding to the same task, and generate a task model corresponding to the same task to obtain the task models corresponding to each of the tasks; wherein each of the tasks includes an emotion recognition task, a topic detection task, and a speaking rate assessment task; the preset neural network model includes a word embedding layer, a position encoding layer, multiple multi-head self-attention layers, multiple residual summation network layers, a feedforward neural network layer, and a fully connected layer.
[0094] In the specific implementation process, the fusion module 2 is specifically used for: performing calculations based on the first model parameter vector and each of the second model parameter vectors to obtain first task vectors corresponding to each task; pruning each of the first task vectors to obtain each second task vector; filtering the model parameters in each of the second task vectors to obtain each third task vector; performing mean merging based on each of the third task vectors to obtain the target model parameter vector of the target speech detection model; constructing a model based on the target model parameter vector using the preset neural network model to obtain the target speech detection model.
[0095] In a specific implementation process, the fusion module 2 is further used to: perform subtraction operations based on each of the second model parameter vectors and the first model parameter vectors, respectively, to obtain first task vectors corresponding to each of the tasks.
[0096] During the specific implementation process, the fusion module 2 is also used to: for the same first task vector, perform absolute value calculation processing on multiple model parameters corresponding to each model structure layer to obtain initial parameter values corresponding to each model structure layer; screen each initial parameter value to obtain each first parameter value and each second parameter value; replace each second parameter value with a numerical value to obtain a third parameter value corresponding to each second parameter value; and reorganize data based on each first parameter value and each third parameter value to obtain the second task vector corresponding to the same first task vector.
[0097] During the specific implementation process, the fusion module 2 is also used to: perform addition operations on the model parameters of each second task vector at the same position of the same model structure layer to obtain model parameters and values corresponding to different positions of the same model structure layer; screen the model parameters of each second task vector based on the model parameters and values corresponding to the same position of the same model structure layer to obtain fourth parameter values and fifth parameter values; replace the numerical value of each fifth parameter value to obtain a sixth parameter value corresponding to each fifth parameter value; and reorganize data based on the fourth parameter values and the sixth parameter values to obtain a third task vector corresponding to each second task vector.
[0098] During the specific implementation process, the fusion module 2 is also used to: perform average calculation processing on the model parameters of each of the third task vectors at the same position in the same model structure layer to obtain an average task vector; perform addition processing on the first model parameter vector of the preset neural network model and the average task vector to obtain a target model parameter vector of the target speech detection model.
[0099] This application adopts the preset neural network model as the basic model, and uses different corpora to train different task types to obtain task models corresponding to different task types; then the task models are merged into a comprehensive target speech detection model by pruning and merging methods, and finally the target speech detection model can be used to detect the speech input text to be detected to obtain the detection results corresponding to each task, thereby improving the flexibility and adaptability of the speech detection method. Since the interference between parameters is considered in the merging process, the performance of the target speech detection model on multiple tasks has been improved without performance loss, making the results of speech quality inspection more accurate.
[0100] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:
[0101] Step 1: Based on each corpus sample, a preset neural network model is used to perform model training for each task to obtain a task model corresponding to each task;
[0102] Step 2: Based on the first model parameter vector of the preset neural network model and the second model parameter vector of each task model, a preset model fusion method is used to fuse each task model to obtain a target speech detection model;
[0103] Step 3: Obtain the voice input text to be detected;
[0104] Step 4: Use the target speech detection model to detect the speech input text to be detected to obtain speech detection results corresponding to each of the tasks.
[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0106] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0107] The specific implementation process of the above method steps can refer to the embodiments of any of the above multi-task speech detection methods, and this embodiment will not be repeated here.
[0108] This application adopts the preset neural network model as the basic model, and uses different corpora to train different task types to obtain task models corresponding to different task types; then the task models are merged into a comprehensive target speech detection model by pruning and merging methods, and finally the target speech detection model can be used to detect the speech input text to be detected to obtain the detection results corresponding to each task, thereby improving the flexibility and adaptability of the speech detection method. Since the interference between parameters is considered in the merging process, the performance of the target speech detection model on multiple tasks has been improved without performance loss, making the results of speech quality inspection more accurate.
[0109] Another embodiment of the present application provides an electronic device, which may be a server, and the electronic device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external client via a network connection. When the electronic device program is executed by the processor, it implements the functions or steps of a multi-task speech detection method server side.
[0110] In one embodiment, an electronic device is provided, which may be a client. The electronic device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external server via a network connection. When the electronic device program is executed by the processor, the functions or steps of the client side of a multi-task speech detection method are implemented.
[0111] Another embodiment of the present application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the following method steps when executing the computer program in the memory:
[0112] Step 1: Based on each corpus sample, a preset neural network model is used to perform model training for each task to obtain a task model corresponding to each task;
[0113] Step 2: Based on the first model parameter vector of the preset neural network model and the second model parameter vector of each task model, a preset model fusion method is used to fuse each task model to obtain a target speech detection model;
[0114] Step 3: Obtain the voice input text to be detected;
[0115] Step 4: Use the target speech detection model to detect the speech input text to be detected to obtain speech detection results corresponding to each of the tasks.
[0116] The specific implementation process of the above method steps can refer to the embodiments of any of the above multi-task speech detection methods, and this embodiment will not be repeated here.
[0117] This application adopts the preset neural network model as the basic model, and uses different corpora to train different task types to obtain task models corresponding to different task types; then the task models are merged into a comprehensive target speech detection model by pruning and merging methods, and finally the target speech detection model can be used to detect the speech input text to be detected to obtain the detection results corresponding to each task, thereby improving the flexibility and adaptability of the speech detection method. Since the interference between parameters is considered in the merging process, the performance of the target speech detection model on multiple tasks has been improved without performance loss, making the results of speech quality inspection more accurate.
[0118] The non-company software tools or components that appear in the embodiments of this application are merely examples and do not represent actual use. The above embodiments are only exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to this application within the essence and scope of protection of this application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of this application.
Claims
1. A multi-task speech detection method, characterized in that: include: Based on each corpus sample, a preset neural network model is used to perform model training for each task respectively to obtain a task model corresponding to each task; Based on the first model parameter vector of the preset neural network model and the second model parameter vector of each of the task models, a preset model fusion method is used to fuse each of the task models to obtain a target speech detection model; Get the voice input text to be detected; The target speech detection model is used to detect the speech input text to be detected, and speech detection results corresponding to each of the tasks are obtained.
2. The method according to claim 1, characterized in that Based on each corpus sample, a preset neural network model is used to perform model training for each task to obtain a task model corresponding to each task, specifically including: Labeling each of the corpus samples based on the task category of each of the tasks to obtain label data corresponding to each of the tasks; For the label data corresponding to the same task, a preset cross entropy loss function is used to perform model training on a preset neural network model to generate a task model corresponding to the same task, so as to obtain the task models corresponding to each of the tasks respectively; The tasks include emotion recognition, topic detection and speech rate assessment. The preset neural network model includes a word embedding layer, a position encoding layer, multiple multi-head self-attention layers, multiple residual summation network layers, a feedforward neural network layer and a fully connected layer.
3. The method according to claim 1, characterized in that The first model parameter vector based on the preset neural network model and the second model parameter vector of each task model are fused by a preset model fusion method to obtain a target speech detection model, specifically comprising: Performing calculations based on the first model parameter vector and each of the second model parameter vectors to obtain a first task vector corresponding to each task; Pruning each of the first task vectors to obtain each second task vector; filtering the model parameters in each of the second task vectors to obtain each third task vector; Perform mean merging based on each of the third task vectors to obtain a target model parameter vector of the target speech detection model; The preset neural network model is used to construct a model based on the target model parameter vector to obtain the target speech detection model.
4. The method according to claim 3, characterized in that The performing calculation based on the first model parameter vector and each of the second model parameter vectors to obtain a first task vector corresponding to each task specifically includes: Subtraction operations are performed on each of the second model parameter vectors and the first model parameter vectors to obtain first task vectors corresponding to each of the tasks.
5. The method according to claim 3, characterized in that The pruning of each of the first task vectors to obtain each of the second task vectors specifically includes: For the same first task vector, performing absolute value calculation processing on a plurality of model parameters corresponding to each model structure layer, to obtain initial parameter values corresponding to each model structure layer; Screening each of the initial parameter values to obtain each first parameter value and each second parameter value; Perform numerical substitution on each of the second parameter values to obtain a third parameter value corresponding to each of the second parameter values; Data is reorganized based on each of the first parameter values and each of the third parameter values to obtain the second task vector corresponding to the same first task vector.
6. The method according to claim 3, characterized in that The filtering of the model parameters in each of the second task vectors to obtain each of the third task vectors specifically includes: Performing an addition operation on the model parameters of each of the second task vectors at the same position in the same model structure layer to obtain model parameters and values corresponding to different positions in the same model structure layer; The model parameters of each of the second task vectors are screened based on the model parameters and values corresponding to the same position of the same model structure layer to obtain each fourth parameter value and each fifth parameter value; Perform numerical substitution on each of the fifth parameter values to obtain a sixth parameter value corresponding to each of the fifth parameter values; Data is reorganized based on each of the fourth parameter values and each of the sixth parameter values to obtain a third task vector corresponding to each of the second task vectors.
7. The method according to claim 3, characterized in that The step of performing mean merging based on each of the third task vectors to obtain a target model parameter vector of the target speech detection model specifically includes: Performing average calculation processing on the model parameters of the third task vectors at the same position in the same model structure layer to obtain an average task vector; An addition operation is performed based on the first model parameter vector of the preset neural network model and the average task vector to obtain a target model parameter vector of the target speech detection model.
8. A multi-task speech detection device, characterized in that: include: A model training module is used to perform model training for each task based on each corpus sample using a preset neural network model to obtain a task model corresponding to each task; A fusion module, used to fuse each of the task models using a preset model fusion method based on the first model parameter vector of the preset neural network model and the second model parameter vector of each of the task models to obtain a target speech detection model; An acquisition module is used to acquire the voice input text to be detected; The detection module is used to detect the speech input text to be detected using the target speech detection model to obtain speech detection results corresponding to each of the tasks.
9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the multi-task speech detection method according to any one of claims 1 to 7 are implemented.
10. An electronic device, characterized in that: The method comprises at least a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the multi-task speech detection method according to any one of claims 1 to 7 when executing the computer program in the memory.