Intelligent voice recognition interaction method and system

Abstract

The invention discloses an intelligent voice recognition interaction method and system, which is used for transmitting user audio data and network state information in real time by establishing network connection between a cloud platform and a user client. And according to the network state, dividing transmission time periods and setting grades, and correspondingly dividing the audio data. The voice recognition model is used for recognizing audio features, feature vectors are reconstructed through the self-encoding network, and the similarity of the audio features is calculated. And setting data credibility based on the state level and the similarity and taking the data credibility as a weight, and extracting second audio transmission data from the real-time audio. And constructing a GAN generation model and importing the second audio data for feature learning, and generating user audio training data for training a speech recognition model, thereby effectively improving the audio recognition effect and improving the interaction efficiency of a speech recognition platform.

Description

Technical Field

[0001] The present invention relates to the field of speech analysis, and more specifically, to an intelligent speech recognition interaction method and system. Background Art

[0002] With the continuous development of speech recognition technology, speech recognition technology has been widely used in conference systems to achieve automatic recording and subsequent processing of conference content. However, the existing conference speech recognition system still has some shortcomings when performing speech recognition. In the real-time transmission and audio analysis of the conference, the conference system (or cloud platform) needs to perform real-time analysis and user identification on a large amount of audio data. This process requires fast and efficient recognition of audio data. Traditional audio interaction analysis technology is often based on a simple audio recognition model with low recognition accuracy, resulting in poor classification of user audio data, and lacks effective data mining and audio recognition model training and optimization processes for real-time audio data. There is also a lack of corresponding training data, which leads to errors in the corresponding audio data recognition and system data entry, reducing the analysis efficiency of the audio analysis platform. Therefore, there is an urgent need for an intelligent audio data analysis and a corresponding method to improve the interactive experience. Summary of the invention

[0003] The present invention overcomes the defects of the prior art and proposes an intelligent speech recognition interaction method and system.

[0004] The first aspect of the present invention provides an intelligent speech recognition interaction method, comprising:

[0005] Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the audio transmission data of the user client within one analysis cycle;

[0006] Acquire network transmission status information within the one analysis period through real-time network transmission, divide multiple transmission time periods based on the network status, set status levels for the multiple transmission time periods, and divide the audio transmission data into corresponding data based on the multiple transmission time periods to form multiple audio data segments;

[0007] Identify the audio data segment through the speech recognition model and extract the user audio features from the system database;

[0008] Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio feature and the reconstructed feature vector data to obtain audio feature similarity;

[0009] According to the status level of each audio data segment and the audio feature similarity, the data credibility is set, and the data credibility is used as the weight of each audio data segment. Based on the weight, data is extracted from the audio transmission data, and the weight is used as the data ratio of each audio data segment in the extraction process to obtain the second audio transmission data;

[0010] Construct a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data;

[0011] The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during the user network conference process.

[0012] In this solution, the cloud platform establishes a network connection with the user client and is applied to the user network conference process. Through real-time network transmission, the audio transmission data of the user client is obtained within an analysis cycle, specifically:

[0013] Based on the preset network transmission protocol, establish a network connection between the cloud platform and the user client;

[0014] Set an analysis cycle and collect data in one analysis cycle;

[0015] During the user network conference, the audio transmission data of the user client is obtained through real-time network transmission, and the audio transmission data is transmitted to the cloud platform.

[0016] In this solution, the network transmission status information within the analysis period is obtained through real-time network transmission, multiple transmission time periods are divided based on the network status, and status levels are set for the multiple transmission time periods. The audio transmission data is divided into corresponding data based on the multiple transmission time periods to form multiple audio data segments, specifically:

[0017] Obtaining network transmission status information within the analysis cycle through real-time network transmission;

[0018] The network transmission status information includes delay, packet loss rate, bandwidth utilization, and traffic statistics information;

[0019] Based on the network transmission status information, a change assessment is performed on the network status of an analysis period, and multiple transmission time periods are divided according to the volatility of the network status;

[0020] A status level is set for each transmission time period, and each transmission time period is a continuous time period.

[0021] In this solution, the audio data segment is identified by the speech recognition model and the user audio features are extracted from the system database, specifically:

[0022] Based on the speech recognition model, audio feature extraction and audio feature recognition are performed on each audio data segment, and user information is marked;

[0023] Based on the user information, user data is retrieved and extracted from the system database to obtain user audio features.

[0024] In this solution, the audio data segment is subjected to audio feature extraction and feature vector conversion to form audio feature vector data, the audio feature vector data is reconstructed through an autoencoder network to generate reconstructed feature vector data, and the user audio feature and the reconstructed feature vector data are subjected to data similarity calculation to obtain audio feature similarity, specifically:

[0025] Extract audio features according to the audio data segment, the audio features include three dimensions: frequency, amplitude, and audio waveform, and convert the extracted features into feature vectors to obtain audio feature vector data;

[0026] Construct an autoencoder network, in which the audio feature vector data is used as high-dimensional input for data mapping through the HLP network, and mapped to low-dimensional representation data;

[0027] Through the decoder, the low-dimensional representation data is reconstructed to obtain high-dimensional representation data;

[0028] Use the high-dimensional representation data as reconstructed feature vector data;

[0029] The difference between the reconstructed feature vector data and the audio feature vector data is judged by the mean square error loss function. If the difference is greater than the expected value, the autoencoder network parameters are updated, and data dimension reduction and data reconstruction are repeated until the expected reconstructed feature vector data is generated. The audio feature similarity at this time and the difference between the reconstructed feature vector data are recorded to obtain the audio feature similarity.

[0030] In this solution, the data credibility is set according to the status level of each audio data segment and the audio feature similarity, and the data credibility is used as the weight of each audio data segment. Based on the weight, data is extracted from the audio transmission data. The extraction process uses the weight as the data ratio of each audio data segment to obtain the second audio transmission data, which is specifically:

[0031] Setting data credibility based on the status level of each audio data segment and the audio feature similarity, where the data credibility is equal to the weighted sum of the status level of the audio data segment and the audio feature similarity;

[0032] Taking the data credibility as the weight of each audio data segment, data extraction is performed from the audio transmission data. Specifically, the data extraction includes calculating the proportion of each audio data segment through the weight, and adjusting the data amount of each audio data segment based on the proportion until the data proportion corresponding to each audio data segment is equal to the proportion;

[0033] Finally, the second audio transmission data is obtained.

[0034] In this solution, the GAN-based generative model is constructed, and the second audio transmission data is imported into the generative model as real data for feature learning and generating user audio training data, specifically:

[0035] Build a GAN-based generative model, which includes a generator and a discriminator;

[0036] Importing the second audio transmission data as real data into the generator for feature learning and simulation data generation;

[0037] Import the simulated data into the discriminator for recognition, and update the parameters of the generator and discriminator based on the preset loss function;

[0038] Perform adversarial training based on the generator and discriminator cycle until the generator and discriminator reach Nash equilibrium;

[0039] A preset amount of simulated data is generated through the trained generative model and marked as user audio training data.

[0040] In this solution, the speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during the user network conference, specifically:

[0041] Import user audio training data into the speech recognition model for model training;

[0042] During the real-time network conference data interaction process, user identification and data interaction are performed based on the speech recognition model.

[0043] The second aspect of the present invention further provides an intelligent speech recognition interaction system, the system comprising: a memory, a processor, the memory comprising an intelligent speech recognition interaction program, the intelligent speech recognition interaction program when executed by the processor to implement the following steps:

[0044] Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the audio transmission data of the user client within one analysis cycle;

[0045] Acquire network transmission status information within the one analysis period through real-time network transmission, divide multiple transmission time periods based on the network status, set status levels for the multiple transmission time periods, and divide the audio transmission data into corresponding data based on the multiple transmission time periods to form multiple audio data segments;

[0046] Identify the audio data segment through the speech recognition model and extract the user audio features from the system database;

[0047] Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio feature and the reconstructed feature vector data to obtain audio feature similarity;

[0048] According to the status level of each audio data segment and the audio feature similarity, the data credibility is set, and the data credibility is used as the weight of each audio data segment. Based on the weight, data is extracted from the audio transmission data, and the weight is used as the data ratio of each audio data segment in the extraction process to obtain the second audio transmission data;

[0049] Construct a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data;

[0050] The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during the user network conference process.

[0051] The third aspect of the present invention also provides a computer-readable storage medium, which includes an intelligent speech recognition interaction program. When the intelligent speech recognition interaction program is executed by a processor, the steps of the intelligent speech recognition interaction method as described in any one of the above items are implemented.

[0052] The present invention discloses an intelligent speech recognition interaction method and system, which transmits user audio data and network status information in real time by establishing a network connection between a cloud platform and a user client. The transmission time period is divided and the level is set according to the network status, and the audio data is divided accordingly. The audio features are identified by a speech recognition model, and the feature vector is reconstructed through an autoencoder network to calculate the audio feature similarity. The data credibility is set based on the status level and similarity and used as a weight to extract the second audio transmission data from the real-time audio. A GAN generation model is constructed and the second audio data is imported for feature learning to generate user audio training data for training the speech recognition model, effectively improving the audio recognition effect and improving the interaction efficiency of the speech recognition platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flow chart of an intelligent speech recognition interaction method of the present invention is shown;

[0054] Figure 2 A block diagram of an intelligent speech recognition interaction system of the present invention is shown. DETAILED DESCRIPTION

[0055] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0057] Figure 1 A flow chart of an intelligent speech recognition interaction method of the present invention is shown.

[0058] like Figure 1 As shown, the first aspect of the present invention provides an intelligent speech recognition interaction method, comprising:

[0059] S102, establishing a network connection between the cloud platform and the user client, and applying it to the user network conference process, obtaining the audio transmission data of the user client within one analysis cycle through real-time network transmission;

[0060] S104, obtaining network transmission status information within the one analysis period through real-time network transmission, dividing a plurality of transmission time periods based on the network status, setting status levels for the plurality of transmission time periods, and performing corresponding data division on the audio transmission data based on the plurality of transmission time periods to form a plurality of audio data segments;

[0061] S106, identifying the audio data segment through a speech recognition model and extracting user audio features from a system database;

[0062] S108, extracting audio features and converting feature vectors from the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio features and the reconstructed feature vector data to obtain audio feature similarity;

[0063] S110, setting data credibility according to the status level of each audio data segment and the audio feature similarity, taking the data credibility as the weight of each audio data segment, extracting data from the audio transmission data based on the weight, and taking the weight as the data ratio of each audio data segment in the extraction process, to obtain second audio transmission data;

[0064] S112, constructing a GAN-based generation model, importing the second audio transmission data as real data into the generation model for feature learning and generating user audio training data;

[0065] S114, training a speech recognition model through user audio training data, and performing user interaction recognition in the user network conference process through the speech recognition model.

[0066] It is worth mentioning that in the real-time transmission and audio analysis of meetings, the conference system (or cloud platform) needs to perform real-time analysis and user identification on a large amount of audio data. This process requires fast and efficient recognition of audio data. Traditional audio interaction analysis technology is often based on simple audio recognition models with low recognition accuracy, resulting in poor classification of user audio data. There is also a lack of effective data mining and audio recognition model training and optimization processes for real-time audio data. There is also often a lack of corresponding training data, which leads to errors in the corresponding audio data recognition and system data entry, reducing the analysis efficiency of the audio analysis platform.

[0067] Based on this, the present invention divides the audio transmission data into multiple data segments, and mines the important features in the audio data through data dimensionality reduction and reconstruction analysis. The audio feature similarity is used as an indicator, and the weight calculation is further combined with the network transmission characteristics. The proportion of each data segment is determined by the weight, and audio features with training significance are further extracted from the audio transmission data. The GAN model is then used for simulation generation, so that better training data can be obtained from real-time transmission, and the speech recognition model can be further optimized.

[0068] Through the present invention, the audio recognition training effect and recognition effect based on the deep learning network are effectively improved, and the training data extraction process can be widely applied to audio platforms or cloud platforms in multiple scenarios, and has wide practicality.

[0069] According to an embodiment of the present invention, the cloud platform establishes a network connection with the user client and is applied to the user network conference process, and obtains the audio transmission data of the user client within an analysis cycle through real-time network transmission, specifically:

[0070] Based on the preset network transmission protocol, establish a network connection between the cloud platform and the user client;

[0071] Set an analysis cycle and collect data in one analysis cycle;

[0072] During the user network conference, the audio transmission data of the user client is obtained through real-time network transmission, and the audio transmission data is transmitted to the cloud platform.

[0073] It should be noted that the cloud platform is used to store, analyze, and identify the collected audio data, and further interact with users.

[0074] According to an embodiment of the present invention, the network transmission status information within the one analysis cycle is obtained through real-time network transmission, multiple transmission time periods are divided based on the network status, and status levels are set for the multiple transmission time periods. The audio transmission data is divided into corresponding data based on the multiple transmission time periods to form multiple audio data segments, specifically:

[0075] Obtaining network transmission status information within the analysis cycle through real-time network transmission;

[0076] The network transmission status information includes delay, packet loss rate, bandwidth usage rate, and traffic statistics information;

[0077] Based on the network transmission status information, a change evaluation is performed on the network status of an analysis cycle, and multiple transmission time periods are divided according to the volatility of the network status;

[0078] A status level is set for each transmission time period, and each transmission time period is a continuous time period.

[0079] It should be noted that, the changes in the network conditions of an analysis period are evaluated, and multiple transmission time periods are divided according to the volatility of the network status. Specifically, the division is based on the transmission quality and fluctuation changes of the network conditions. The multiple transmission time periods obtained have the characteristic of large differences in network conditions.

[0080] According to an embodiment of the present invention, the audio data segment is identified by a speech recognition model and user audio features are extracted from a system database, specifically:

[0081] Based on the speech recognition model, audio feature extraction and audio feature recognition are performed on each audio data segment, and user information is marked;

[0082] Based on the user information, user data is retrieved and extracted from the system database to obtain user audio features.

[0083] It should be noted that the speech recognition model is a recognition model based on deep learning, which performs user speech recognition and corresponding interactive functions by learning audio features. The user audio features are the preferred feature data stored in the database, which are used to compare with the audio data obtained in real time in the future to analyze the credibility of the corresponding data.

[0084] According to an embodiment of the present invention, the audio data segment is subjected to audio feature extraction and feature vector conversion to form audio feature vector data, the audio feature vector data is reconstructed through an autoencoder network to generate reconstructed feature vector data, and the user audio feature and the reconstructed feature vector data are subjected to data similarity calculation to obtain audio feature similarity, specifically:

[0085] Extract audio features based on the audio data segment. The audio features include three dimensions: frequency, amplitude, and audio waveform. Convert the extracted features into feature vectors to obtain audio feature vector data.

[0086] Construct an autoencoder network, in which the audio feature vector data is used as high-dimensional input for data mapping through the MLP network, and mapped to low-dimensional representation data;

[0087] Through the decoder, the low-dimensional representation data is reconstructed to obtain high-dimensional representation data;

[0088] Use the high-dimensional representation data as reconstructed feature vector data;

[0089] The difference between the reconstructed feature vector data and the audio feature vector data is judged by the mean square error loss function. If the difference is greater than the expected value, the autoencoder network parameters are updated, and data dimension reduction and data reconstruction are repeated until the expected reconstructed feature vector data is generated. The audio feature similarity at this time and the difference between the reconstructed feature vector data are recorded to obtain the audio feature similarity.

[0090] It should be noted that the MLP network is a multi-layer perceptron. The expected value is the user-set value. Judging the difference is judging the similarity. The audio feature similarity reflects the difference between the audio data segment and the standard user audio feature under the low-dimensional representation. The greater the similarity, the higher the accuracy of the data segment for the subsequent simulated audio feature.

[0091] According to an embodiment of the present invention, the data credibility is set according to the status level of each audio data segment and the audio feature similarity, the data credibility is used as the weight of each audio data segment, and data is extracted from the audio transmission data based on the weight. The extraction process uses the weight as the data ratio of each audio data segment to obtain the second audio transmission data, which is specifically:

[0092] The data credibility is set based on the status level of each audio data segment and the audio feature similarity, and the data credibility is equal to the weighted sum of the status level of the audio data segment and the audio feature similarity;

[0093] Taking the data credibility as the weight of each audio data segment, data extraction is performed from the audio transmission data. Specifically, the data extraction includes calculating the proportion of each audio data segment through the weight, and adjusting the data amount of each audio data segment based on the proportion until the data proportion corresponding to each audio data segment is equal to the proportion;

[0094] Finally, the second audio transmission data is obtained.

[0095] The calculation expression of data credibility R is as follows:

[0096] R = L × K1 + S × K2;

[0097] L is the status level, S is the audio feature similarity. K1 and K2 are preset weight values. By changing the weight value, the initially screened second audio transmission data can be adjusted, and the correlation between the status level of the corresponding data and the audio feature similarity can be freely set.

[0098] It should be noted that the proportion is proportional to the weight of the audio data segment. The data volume of each audio data segment is adjusted, including increasing and decreasing data. When decreasing data, data is randomly deleted. When increasing data, a certain segment of data is randomly set in the audio data segment for duplication to increase the data volume of the audio data segment.

[0099] According to an embodiment of the present invention, the construction of a GAN-based generation model, importing the second audio transmission data as real data into the generation model for feature learning and generating user audio training data, is specifically as follows:

[0100] Build a GAN-based generative model, which includes a generator and a discriminator;

[0101] Import the second audio transmission data as real data into the generator for feature learning and simulation data generation;

[0102] Import the simulated data into the discriminator for recognition, and update the parameters of the generator and discriminator based on the preset loss function;

[0103] Perform adversarial training based on the generator and discriminator cycle until the generator and discriminator reach Nash equilibrium;

[0104] A preset amount of simulated data is generated through the trained generative model and marked as user audio training data.

[0105] According to an embodiment of the present invention, the speech recognition model is trained by using user audio training data, and the user interaction recognition during the user network conference is performed by using the speech recognition model, specifically:

[0106] Import user audio training data into the speech recognition model for model training;

[0107] During the real-time network conference data interaction process, user identification and data interaction are performed based on the speech recognition model.

[0108] According to an embodiment of the present invention, it also includes:

[0109] Through real-time network transmission, the real-time audio transmission data of the user client is obtained in a real-time cycle;

[0110] Based on the network status, the real-time audio transmission data is divided into a plurality of transmission time periods and a plurality of corresponding real-time data segments;

[0111] Perform feature vector conversion on real-time data segments and reconstruct data through an autoencoder network to obtain real-time reconstruction features;

[0112] Calculate the similarity between the real-time data segment and the real-time reconstructed feature data based on the standard Euclidean distance;

[0113] Analyzing multiple real-time data segments to obtain multiple similarities;

[0114] The data compression rate is set based on the similarity. The higher the similarity, the higher the data compression rate. According to the data compression rate, the audio compression algorithm is dynamically selected for each real-time data segment in the preset compression algorithm, and a compression algorithm transmission scheme is generated;

[0115] Based on the compression algorithm transmission solution, audio data compression transmission and data interaction are performed between the cloud platform and the user client.

[0116] It should be noted that the real-time period is a relatively short period of time. Each real-time data segment corresponds to a real-time reconstruction feature.

[0117] In the present invention, an autoencoding network is utilized to analyze the dimension reduction process of real-time interactive audio data, and a similarity analysis is performed on the data before and after the output of the autoencoding network. The higher the similarity, the more important the corresponding audio feature is to the speech recognition model. Based on this, the compression rate is evaluated. The higher the similarity, the higher the compression rate. This ensures that the speech recognition model can effectively compress the audio data while maintaining a high recognition rate, reduce the amount of data transmission, and improve the real-time transmission efficiency of the audio data. A dynamic planning compression scheme is implemented to further improve the real-time interaction capability between the cloud platform and the user client.

[0118] According to an embodiment of the present invention, dividing a plurality of transmission time periods based on the network status further includes:

[0119] The complexity of the current network status is determined by the network transmission status information. If the complexity of the network status is higher than a predetermined level, multiple unit time periods are divided based on an analysis cycle.

[0120] Through the network transmission status information, based on multiple dimensions such as delay, packet loss rate, bandwidth utilization rate, and traffic statistics, corresponding network feature data is generated for each unit time period;

[0121] Using K-means clustering, multiple network feature data are clustered, and unit time periods are grouped to form K time groups;

[0122] The unit time periods in each time group are combined to generate a transmission time period, and K transmission time periods are generated based on the K time groups.

[0123] It should be noted that the complexity of the network condition is comprehensively evaluated by combining delay, packet loss rate, bandwidth utilization rate, and traffic statistics. The higher the complexity of the network condition, the more unstable the network status. In the process of dividing the transmission time period, affected by the complexity of the network, the corresponding division can be performed based on the clustering algorithm to obtain time periods with a high classification effect, so as to efficiently screen and extract data in the subsequent period.

[0124] Figure 2 A block diagram of an intelligent speech recognition interaction system of the present invention is shown.

[0125] The second aspect of the present invention further provides an intelligent voice recognition interaction system 2, which includes: a memory 21 and a processor 22, wherein the memory 21 includes an intelligent voice recognition interaction program, and when the intelligent voice recognition interaction program is executed by the processor 22, the following steps are implemented:

[0126] Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the audio transmission data of the user client within one analysis cycle;

[0127] Acquire network transmission status information within the one analysis period through real-time network transmission, divide multiple transmission time periods based on the network status, set status levels for the multiple transmission time periods, and divide the audio transmission data into corresponding data based on the multiple transmission time periods to form multiple audio data segments;

[0128] Identify the audio data segment through the speech recognition model and extract the user audio features from the system database;

[0129] Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio feature and the reconstructed feature vector data to obtain audio feature similarity;

[0130] According to the status level of each audio data segment and the audio feature similarity, the data credibility is set, and the data credibility is used as the weight of each audio data segment. Based on the weight, data is extracted from the audio transmission data, and the weight is used as the data ratio of each audio data segment in the extraction process to obtain the second audio transmission data;

[0131] Construct a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data;

[0132] The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during the user network conference process.

[0133] It is worth mentioning that in the real-time transmission and audio analysis of meetings, the conference system (or cloud platform) needs to perform real-time analysis and user identification on a large amount of audio data. This process requires fast and efficient recognition of audio data. Traditional audio interaction analysis technology is often based on simple audio recognition models with low recognition accuracy, resulting in poor classification of user audio data. There is also a lack of effective data mining and audio recognition model training and optimization processes for real-time audio data. There is also often a lack of corresponding training data, which leads to errors in the corresponding audio data recognition and system data entry, reducing the analysis efficiency of the audio analysis platform.

[0134] Based on this, the present invention divides the audio transmission data into multiple data segments, and mines the important features in the audio data through data dimensionality reduction and reconstruction analysis. The audio feature similarity is used as an indicator, and the weight calculation is further combined with the network transmission characteristics. The proportion of each data segment is determined by the weight, and audio features with training significance are further extracted from the audio transmission data. The GAN model is then used for simulation generation, so that better training data can be obtained from real-time transmission, and the speech recognition model can be further optimized.

[0135] Through the present invention, the audio recognition training effect and recognition effect based on the deep learning network are effectively improved, and the training data extraction process can be widely applied to audio platforms or cloud platforms in multiple scenarios, and has wide practicality.

[0136] According to an embodiment of the present invention, the cloud platform establishes a network connection with the user client and is applied to the user network conference process, and obtains the audio transmission data of the user client within an analysis cycle through real-time network transmission, specifically:

[0137] Based on the preset network transmission protocol, establish a network connection between the cloud platform and the user client;

[0138] Set an analysis cycle and collect data in one analysis cycle;

[0139] During the user network conference, the audio transmission data of the user client is obtained through real-time network transmission, and the audio transmission data is transmitted to the cloud platform.

[0140] It should be noted that the cloud platform is used to store, analyze, and identify the collected audio data, and further interact with users.

[0141] According to an embodiment of the present invention, the network transmission status information within the one analysis cycle is obtained through real-time network transmission, multiple transmission time periods are divided based on the network status, and status levels are set for the multiple transmission time periods. The audio transmission data is divided into corresponding data based on the multiple transmission time periods to form multiple audio data segments, specifically:

[0142] Obtaining network transmission status information within the analysis cycle through real-time network transmission;

[0143] The network transmission status information includes delay, packet loss rate, bandwidth utilization, and traffic statistics information;

[0144] Based on the network transmission status information, a change assessment is performed on the network status of an analysis period, and multiple transmission time periods are divided according to the volatility of the network status;

[0145] A status level is set for each transmission time period, and each transmission time period is a continuous time period.

[0146] It should be noted that, the changes in the network conditions of an analysis period are evaluated, and multiple transmission time periods are divided according to the volatility of the network status. Specifically, the division is based on the transmission quality and fluctuation changes of the network conditions. The multiple transmission time periods obtained have the characteristic of large differences in network conditions.

[0147] According to an embodiment of the present invention, the audio data segment is identified by a speech recognition model and user audio features are extracted from a system database, specifically:

[0148] Based on the speech recognition model, audio feature extraction and audio feature recognition are performed on each audio data segment, and user information is marked;

[0149] Based on the user information, user data is retrieved and extracted from the system database to obtain user audio features.

[0150] It should be noted that the speech recognition model is a recognition model based on deep learning, which performs user speech recognition and corresponding interactive functions by learning audio features. The user audio features are the preferred feature data stored in the database, which are used to compare with the audio data obtained in real time in the future to analyze the credibility of the corresponding data.

[0151] According to an embodiment of the present invention, the audio data segment is subjected to audio feature extraction and feature vector conversion to form audio feature vector data, the audio feature vector data is reconstructed through an autoencoder network to generate reconstructed feature vector data, and the user audio feature and the reconstructed feature vector data are subjected to data similarity calculation to obtain audio feature similarity, specifically:

[0152] Extract audio features based on the audio data segment. The audio features include three dimensions: frequency, amplitude, and audio waveform. Convert the extracted features into feature vectors to obtain audio feature vector data.

[0153] Construct an autoencoder network, in which the audio feature vector data is used as high-dimensional input for data mapping through the MLP network, and mapped to low-dimensional representation data;

[0154] Through the decoder, the low-dimensional representation data is reconstructed to obtain high-dimensional representation data;

[0155] Use the high-dimensional representation data as reconstructed feature vector data;

[0156] The difference between the reconstructed feature vector data and the audio feature vector data is judged by the mean square error loss function. If the difference is greater than the expected value, the autoencoder network parameters are updated, and data dimension reduction and data reconstruction are repeated until the expected reconstructed feature vector data is generated. The audio feature similarity at this time and the difference between the reconstructed feature vector data are recorded to obtain the audio feature similarity.

[0157] It should be noted that the MLP network is a multi-layer perceptron. The expected value is the user-set value. Judging the difference is judging the similarity. The audio feature similarity reflects the difference between the audio data segment and the standard user audio feature under the low-dimensional representation. The greater the similarity, the higher the accuracy of the data segment for the subsequent simulated audio feature.

[0158] According to an embodiment of the present invention, the data credibility is set according to the status level of each audio data segment and the audio feature similarity, the data credibility is used as the weight of each audio data segment, and data is extracted from the audio transmission data based on the weight. The extraction process uses the weight as the data ratio of each audio data segment to obtain the second audio transmission data, which is specifically:

[0159] Setting data credibility based on the status level of each audio data segment and the audio feature similarity, where the data credibility is equal to the weighted sum of the status level of the audio data segment and the audio feature similarity;

[0160] Taking the data credibility as the weight of each audio data segment, data extraction is performed from the audio transmission data. Specifically, the data extraction includes calculating the proportion of each audio data segment through the weight, and adjusting the data amount of each audio data segment based on the proportion until the data proportion corresponding to each audio data segment is equal to the proportion;

[0161] Finally, the second audio transmission data is obtained.

[0162] The calculation expression of data credibility R is as follows:

[0163] R = L × K1 + S × K2;

[0164] L is the status level, S is the audio feature similarity. K1 and K2 are preset weight values. By changing the weight value, the initially screened second audio transmission data can be adjusted, and the correlation between the status level of the corresponding data and the audio feature similarity can be freely set.

[0165] It should be noted that the proportion is proportional to the weight of the audio data segment. The data volume of each audio data segment is adjusted, including increasing and decreasing data. When decreasing data, data is randomly deleted. When increasing data, a certain segment of data is randomly set in the audio data segment for duplication to increase the data volume of the audio data segment.

[0166] According to an embodiment of the present invention, the construction of a GAN-based generation model, importing the second audio transmission data as real data into the generation model for feature learning and generating user audio training data, is specifically as follows:

[0167] Build a GAN-based generative model, which includes a generator and a discriminator;

[0168] Importing the second audio transmission data as real data into the generator for feature learning and simulation data generation;

[0169] Import the simulated data into the discriminator for recognition, and update the parameters of the generator and discriminator based on the preset loss function;

[0170] Perform adversarial training based on the generator and discriminator cycle until the generator and discriminator reach Nash equilibrium;

[0171] A preset amount of simulated data is generated through the trained generative model and marked as user audio training data.

[0172] According to an embodiment of the present invention, the speech recognition model is trained by using user audio training data, and the user interaction recognition during the user network conference is performed by using the speech recognition model, specifically:

[0173] Import user audio training data into the speech recognition model for model training;

[0174] During the real-time network conference data interaction process, user identification and data interaction are performed based on the speech recognition model.

[0175] The third aspect of the present invention also provides a computer-readable storage medium, which includes an intelligent speech recognition interaction program. When the intelligent speech recognition interaction program is executed by a processor, the steps of the intelligent speech recognition interaction method as described in any one of the above items are implemented.

[0176] The present invention discloses an intelligent speech recognition interaction method and system, which transmits user audio data and network status information in real time by establishing a network connection between a cloud platform and a user client. The transmission time period is divided and the level is set according to the network status, and the audio data is divided accordingly. The audio features are identified by a speech recognition model, and the feature vector is reconstructed through an autoencoder network to calculate the audio feature similarity. The data credibility is set based on the status level and similarity and used as a weight to extract the second audio transmission data from the real-time audio. A GAN generation model is constructed and the second audio data is imported for feature learning to generate user audio training data for training the speech recognition model, effectively improving the audio recognition effect and improving the interaction efficiency of the speech recognition platform.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0178] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0179] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0180] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0181] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0182] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An intelligent speech recognition interaction method, characterized in that: include: Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the audio transmission data of the user client within one analysis cycle; Acquire network transmission status information within the one analysis period through real-time network transmission, divide multiple transmission time periods based on the network status, set status levels for the multiple transmission time periods, and divide the audio transmission data into corresponding data based on the multiple transmission time periods to form multiple audio data segments; Identify the audio data segment through the speech recognition model and extract the user audio features from the system database; Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio feature and the reconstructed feature vector data to obtain audio feature similarity; According to the status level of each audio data segment and the audio feature similarity, the data credibility is set, and the data credibility is used as the weight of each audio data segment. Based on the weight, data is extracted from the audio transmission data, and the weight is used as the data ratio of each audio data segment in the extraction process to obtain the second audio transmission data; Construct a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data; The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during the user network conference process.

2. The intelligent speech recognition interaction method according to claim 1, characterized in that: The cloud platform establishes a network connection with the user client and is applied to the user network conference process. Through real-time network transmission, the audio transmission data of the user client is obtained within an analysis cycle, specifically: Based on the preset network transmission protocol, establish a network connection between the cloud platform and the user client; Set an analysis cycle and collect data in one analysis cycle; During the user network conference, the audio transmission data of the user client is obtained through real-time network transmission, and the audio transmission data is transmitted to the cloud platform.

3. The intelligent speech recognition interaction method according to claim 2, characterized in that: The real-time network transmission is used to obtain the network transmission status information within the analysis period, multiple transmission time periods are divided based on the network status, and status levels are set for the multiple transmission time periods. The audio transmission data is divided into corresponding data based on the multiple transmission time periods to form multiple audio data segments, specifically: Acquiring network transmission status information within the one analysis cycle through real-time network transmission; The network transmission status information includes delay, packet loss rate, bandwidth usage rate, and traffic statistics information; Based on the network transmission status information, a change assessment is performed on the network status of an analysis period, and multiple transmission time periods are divided according to the volatility of the network status; A status level is set for each transmission time period, and each transmission time period is a continuous time period.

4. The intelligent speech recognition interaction method according to claim 3, characterized in that: The audio data segment is identified by the speech recognition model and the user audio features are extracted from the system database, specifically: Based on the speech recognition model, audio feature extraction and audio feature recognition are performed on each audio data segment, and user information is marked; Based on the user information, user data is retrieved and extracted from the system database to obtain user audio features.

5. The intelligent speech recognition interaction method according to claim 4, characterized in that: The audio data segment is subjected to audio feature extraction and feature vector conversion to form audio feature vector data, the audio feature vector data is subjected to data reconstruction through an autoencoder network to generate reconstructed feature vector data, and the user audio feature and the reconstructed feature vector data are subjected to data similarity calculation to obtain audio feature similarity, specifically: Extract audio features according to the audio data segment, the audio features include three dimensions: frequency, amplitude, and audio waveform, and convert the extracted features into feature vectors to obtain audio feature vector data; Construct an autoencoder network, in which the audio feature vector data is used as high-dimensional input for data mapping through the MLP network, and mapped to low-dimensional representation data; Through the decoder, the low-dimensional representation data is reconstructed to obtain high-dimensional representation data; Use the high-dimensional representation data as reconstructed feature vector data; The difference between the reconstructed feature vector data and the audio feature vector data is judged by the mean square error loss function. If the difference is greater than the expected value, the autoencoder network parameters are updated, and data dimension reduction and data reconstruction are repeated until the expected reconstructed feature vector data is generated. The audio feature similarity at this time and the difference between the reconstructed feature vector data are recorded to obtain the audio feature similarity.

6. The intelligent speech recognition interaction method according to claim 5, characterized in that: The data credibility is set according to the status level of each audio data segment and the audio feature similarity, the data credibility is used as the weight of each audio data segment, and data is extracted from the audio transmission data based on the weight. The extraction process uses the weight as the data ratio of each audio data segment to obtain the second audio transmission data, which is specifically: Setting data credibility based on the status level of each audio data segment and the audio feature similarity, where the data credibility is equal to the weighted sum of the status level of the audio data segment and the audio feature similarity; Taking the data credibility as the weight of each audio data segment, data extraction is performed from the audio transmission data. Specifically, the data extraction includes calculating the proportion of each audio data segment through the weight, and adjusting the data amount of each audio data segment based on the proportion until the data proportion corresponding to each audio data segment is equal to the proportion; Finally, the second audio transmission data is obtained.

7. The intelligent speech recognition interaction method according to claim 6, characterized in that: The construction of the GAN-based generation model, importing the second audio transmission data as real data into the generation model for feature learning and generating user audio training data, is specifically as follows: Build a GAN-based generative model, which includes a generator and a discriminator; Importing the second audio transmission data as real data into the generator for feature learning and simulation data generation; Import the simulated data into the discriminator for recognition, and update the parameters of the generator and discriminator based on the preset loss function; Perform adversarial training based on the generator and discriminator cycle until the generator and discriminator reach Nash equilibrium; A preset amount of simulated data is generated through the trained generative model and marked as user audio training data.

8. The intelligent speech recognition interaction method according to claim 7, characterized in that: The speech recognition model is trained by using the user audio training data, and the user interaction recognition during the user network conference is performed by using the speech recognition model, specifically: Import user audio training data into the speech recognition model for model training; During the real-time network conference data interaction process, user identification and data interaction are performed based on the speech recognition model.

9. An intelligent speech recognition interactive system, characterized in that: The system includes: a memory and a processor, wherein the memory includes an intelligent speech recognition interaction program, and when the intelligent speech recognition interaction program is executed by the processor, the following steps are implemented: Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the audio transmission data of the user client within one analysis cycle; Acquire network transmission status information within the one analysis period through real-time network transmission, divide multiple transmission time periods based on the network status, set status levels for the multiple transmission time periods, and divide the audio transmission data into corresponding data based on the multiple transmission time periods to form multiple audio data segments; Identify the audio data segment through the speech recognition model and extract the user audio features from the system database; Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio feature and the reconstructed feature vector data to obtain audio feature similarity; According to the status level of each audio data segment and the audio feature similarity, the data credibility is set, and the data credibility is used as the weight of each audio data segment. Based on the weight, data is extracted from the audio transmission data, and the weight is used as the data ratio of each audio data segment in the extraction process to obtain the second audio transmission data; Construct a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data; The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during the user network conference process.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes an intelligent speech recognition interaction program, and when the intelligent speech recognition interaction program is executed by a processor, the steps of the intelligent speech recognition interaction method according to any one of claims 1 to 8 are implemented.

Images