Intelligent voice recognition interaction method and system

Abstract

The present invention discloses an intelligent speech recognition interaction method and system, which transmits user audio data and network status information in real time by establishing a network connection between a cloud platform and a user client. The transmission time period is divided and the level is set according to the network status, and the audio data is divided accordingly. The speech recognition model is used to identify audio features, and the feature vector is reconstructed through the autoencoding network to calculate the similarity of the audio features. The data credibility is set based on the status level and similarity and used as a weight to extract the second audio transmission data from the real-time audio. A GAN generation model is constructed and the second audio data is imported for feature learning to generate user audio training data for training the speech recognition model, effectively improving the audio recognition effect and improving the interaction efficiency of the speech recognition platform.

Description

Technical Field

[0001] The present invention relates to the field of speech analysis, and more specifically, to an intelligent speech recognition interaction method and system. Background Art

[0002] With the continuous development of speech recognition technology, it has been widely adopted in conferencing systems, enabling the automatic recording and subsequent processing of meeting content. However, existing conferencing speech recognition systems still have some shortcomings. In real-time conference transmission and audio analysis, conferencing systems (or cloud platforms) need to analyze and identify large amounts of audio data in real time. This process requires fast and efficient audio data recognition. Traditional audio interaction analysis technologies often rely on simple audio recognition models, resulting in low recognition accuracy and poor classification of user audio data. Furthermore, they lack effective data mining and audio recognition model training and optimization processes for real-time audio data. Furthermore, a lack of appropriate training data leads to errors in audio data recognition and system data entry, reducing the efficiency of audio analysis platforms. Therefore, there is an urgent need for intelligent audio data analysis and methods to improve the interactive experience. Summary of the Invention

[0003] The present invention overcomes the defects of the prior art and proposes an intelligent speech recognition interaction method and system.

[0004] A first aspect of the present invention provides an intelligent speech recognition interaction method, comprising:

[0005] Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the user client's audio transmission data within one analysis cycle;

[0006] Acquiring network transmission status information within the analysis period through real-time network transmission, dividing a plurality of transmission time periods based on the network status, setting status levels for the plurality of transmission time periods, and dividing the audio transmission data accordingly based on the plurality of transmission time periods to form a plurality of audio data segments;

[0007] Identify the audio data segment through the speech recognition model and extract the user audio features from the system database;

[0008] Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio features and the reconstructed feature vector data to obtain audio feature similarity;

[0009] Setting data credibility based on the status level of each audio data segment and the similarity between the audio features, using the data credibility as a weight for each audio data segment, and extracting data from the audio transmission data based on the weight, using the weight as a data ratio for each audio data segment in the extraction process, to obtain second audio transmission data;

[0010] Build a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data;

[0011] The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during user network conferences.

[0012] In this solution, the cloud platform establishes a network connection with the user client and is applied to the user network conference process. Through real-time network transmission, the audio transmission data of the user client is obtained within an analysis cycle. Specifically:

[0013] Based on the preset network transmission protocol, establish a network connection between the cloud platform and the user client;

[0014] Set an analysis cycle and collect data in one analysis cycle;

[0015] During the user's online conference, the audio transmission data of the user client is obtained through real-time network transmission and transmitted to the cloud platform.

[0016] In this solution, the network transmission status information within the analysis period is obtained through real-time network transmission, multiple transmission time periods are divided based on the network status, and status levels are set for the multiple transmission time periods. The audio transmission data is divided accordingly based on the multiple transmission time periods to form multiple audio data segments, specifically:

[0017] Acquiring network transmission status information within the analysis period through real-time network transmission;

[0018] The network transmission status information includes delay, packet loss rate, bandwidth usage, and traffic statistics;

[0019] Based on the network transmission status information, a change assessment of the network status in an analysis period is performed, and multiple transmission time periods are divided according to the volatility of the network status;

[0020] A status level is set for each transmission time period, and each transmission time period is a continuous time period.

[0021] In this solution, the audio data segment is identified by the speech recognition model and the user audio features are extracted from the system database, specifically:

[0022] Perform audio feature extraction and audio feature recognition on each audio data segment based on the speech recognition model, and mark the user information;

[0023] Based on user information, user data is retrieved and extracted from the system database to obtain user audio features.

[0024] In this solution, the audio data segment is subjected to audio feature extraction and feature vector conversion to form audio feature vector data, the audio feature vector data is reconstructed through an autoencoder network to generate reconstructed feature vector data, and the user audio feature and the reconstructed feature vector data are subjected to data similarity calculation to obtain audio feature similarity, specifically:

[0025] Extract audio features based on the audio data segment. The audio features include three dimensions: frequency, amplitude, and audio waveform. The extracted features are converted into feature vectors to obtain audio feature vector data.

[0026] Construct an autoencoder network, in which the audio feature vector data is used as high-dimensional input for data mapping through the MLP network, and mapped to low-dimensional representation data;

[0027] Through the decoder, the low-dimensional representation data is reconstructed to obtain high-dimensional representation data;

[0028] Use high-dimensional representation data as reconstructed feature vector data;

[0029] The difference between the reconstructed feature vector data and the audio feature vector data is judged by the mean square error loss function. If the difference is greater than the expected value, the autoencoder network parameters are updated, and data dimensionality reduction and data reconstruction are repeated until the expected reconstructed feature vector data is generated. The difference between the audio feature similarity at this time and the reconstructed feature vector data is recorded to obtain the audio feature similarity.

[0030] In this solution, the data credibility is set according to the status level of each audio data segment and the similarity between the audio features, and the data credibility is used as the weight of each audio data segment. Based on the weight, data is extracted from the audio transmission data. The extraction process uses the weight as the data ratio of each audio data segment to obtain the second audio transmission data, which is specifically:

[0031] Setting data credibility based on the status level of each audio data segment and the audio feature similarity, where the data credibility is equal to the weighted sum of the status level of the audio data segment and the audio feature similarity;

[0032] Using data credibility as the weight of each audio data segment, data extraction is performed from the audio transmission data. Specifically, the data extraction includes calculating the proportion of each audio data segment through the weight, and adjusting the data volume of each audio data segment based on the proportion until the data proportion corresponding to each audio data segment is equal to the proportion;

[0033] Finally, the second audio transmission data is obtained.

[0034] In this solution, the GAN-based generative model is constructed, and the second audio transmission data is imported into the generative model as real data for feature learning and generation of user audio training data, specifically:

[0035] Build a GAN-based generative model, which includes a generator and a discriminator;

[0036] Importing the second audio transmission data as real data into the generator for feature learning and simulation data generation;

[0037] Import the simulated data into the discriminator for recognition, and update the parameters of the generator and discriminator based on the preset loss function;

[0038] Conduct adversarial training based on the generator and discriminator cycles until the generator and discriminator reach Nash equilibrium;

[0039] A preset amount of simulated data is generated through the trained generative model and marked as user audio training data.

[0040] In this solution, the speech recognition model is trained using user audio training data, and the speech recognition model is used to identify user interactions during the user network conference. Specifically,

[0041] Import user audio training data into the speech recognition model for model training;

[0042] During the real-time web conference data interaction process, user identification and data interaction are performed based on the speech recognition model.

[0043] A second aspect of the present invention further provides an intelligent speech recognition interaction system, the system comprising: a memory and a processor, the memory comprising an intelligent speech recognition interaction program, the intelligent speech recognition interaction program implementing the following steps when executed by the processor:

[0044] Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the user client's audio transmission data within one analysis cycle;

[0045] Acquiring network transmission status information within the analysis period through real-time network transmission, dividing a plurality of transmission time periods based on the network status, setting status levels for the plurality of transmission time periods, and dividing the audio transmission data accordingly based on the plurality of transmission time periods to form a plurality of audio data segments;

[0046] Identify the audio data segment through the speech recognition model and extract the user audio features from the system database;

[0047] Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio features and the reconstructed feature vector data to obtain audio feature similarity;

[0048] Setting data credibility based on the status level of each audio data segment and the similarity between the audio features, using the data credibility as a weight for each audio data segment, and extracting data from the audio transmission data based on the weight, using the weight as a data ratio for each audio data segment in the extraction process, to obtain second audio transmission data;

[0049] Build a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data;

[0050] The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during user network conferences.

[0051] The third aspect of the present invention also provides a computer-readable storage medium, which includes an intelligent voice recognition interaction program. When the intelligent voice recognition interaction program is executed by a processor, the steps of the intelligent voice recognition interaction method as described in any one of the above items are implemented.

[0052] The present invention discloses an intelligent speech recognition interaction method and system, which transmits user audio data and network status information in real time by establishing a network connection between a cloud platform and a user client. The transmission time period is divided and the level is set according to the network status, and the audio data is divided accordingly. The speech recognition model is used to identify audio features, and the feature vector is reconstructed through the autoencoding network to calculate the similarity of the audio features. The data credibility is set based on the status level and similarity and used as a weight to extract the second audio transmission data from the real-time audio. A GAN generation model is constructed and the second audio data is imported for feature learning to generate user audio training data for training the speech recognition model, effectively improving the audio recognition effect and improving the interaction efficiency of the speech recognition platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flow chart of an intelligent speech recognition interaction method of the present invention is shown;

[0054] Figure 2 A block diagram of an intelligent speech recognition interaction system of the present invention is shown. DETAILED DESCRIPTION

[0055] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0057] Figure 1 A flow chart of an intelligent speech recognition interaction method of the present invention is shown.

[0058] like Figure 1 As shown, the first aspect of the present invention provides an intelligent speech recognition interaction method, comprising:

[0059] S102, establishing a network connection between the cloud platform and the user client, and applying it to the user network conference process, obtaining the user client's audio transmission data through real-time network transmission within an analysis cycle;

[0060] S104, obtaining network transmission status information within the one analysis period through real-time network transmission, dividing a plurality of transmission time periods based on the network status, setting status levels for the plurality of transmission time periods, and dividing the audio transmission data accordingly based on the plurality of transmission time periods to form a plurality of audio data segments;

[0061] S106, identifying the audio data segment using a speech recognition model and extracting user audio features from a system database;

[0062] S108, extracting audio features and converting feature vectors into audio feature vectors for the audio data segments to form audio feature vector data, reconstructing the audio feature vector data using an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio features and the reconstructed feature vector data to obtain audio feature similarity;

[0063] S110, setting data credibility based on the status level of each audio data segment and the audio feature similarity, using the data credibility as a weight for each audio data segment, and extracting data from the audio transmission data based on the weight, using the weight as a data ratio for each audio data segment in the extraction process, to obtain second audio transmission data;

[0064] S112, building a generative model based on GAN, importing the second audio transmission data as real data into the generative model for feature learning and generating user audio training data;

[0065] S114, training a speech recognition model using user audio training data, and performing user interaction recognition during the user network conference using the speech recognition model.

[0066] It is worth mentioning that in the real-time transmission and audio analysis of conferences, the conference system (or cloud platform) needs to perform real-time analysis and user identification on a large amount of audio data. This process requires fast and efficient recognition of audio data. Traditional audio interaction analysis technology is often based on simple audio recognition models with low recognition accuracy, resulting in poor classification of user audio data. There is also a lack of effective data mining and audio recognition model training and optimization processes for real-time audio data. There is also often a lack of corresponding training data, which leads to errors in the corresponding audio data recognition and system data entry, reducing the analysis efficiency of the audio analysis platform.

[0067] Based on this, the present invention divides the audio transmission data into multiple data segments, and through data dimensionality reduction and reconstruction analysis, it mines the important features in the audio data, and uses the audio feature similarity as an indicator. It further combines the network transmission characteristics to perform weight calculation, and determines the proportion of each data segment by the weight. It further extracts audio features with training significance from the audio transmission data, and subsequently simulates and generates them through the GAN model, so that better training data can be obtained from real-time transmission, and the speech recognition model can be further optimized.

[0068] Through the present invention, the audio recognition training effect and recognition effect based on the deep learning network are effectively improved, and the training data extraction process can be widely applied to audio platforms or cloud platforms in multiple scenarios, and has wide practicality.

[0069] According to an embodiment of the present invention, the cloud platform establishes a network connection with the user client and is applied to the user network conference process. Through real-time network transmission, the audio transmission data of the user client is obtained within an analysis cycle, specifically:

[0070] Based on the preset network transmission protocol, establish a network connection between the cloud platform and the user client;

[0071] Set an analysis cycle and collect data in one analysis cycle;

[0072] During the user's online conference, the audio transmission data of the user client is obtained through real-time network transmission and transmitted to the cloud platform.

[0073] It should be noted that the cloud platform is used to store, analyze, and identify the collected audio data, and further interact with users.

[0074] According to an embodiment of the present invention, the network transmission status information within the analysis period is obtained through real-time network transmission, multiple transmission time periods are divided based on the network status, and status levels are set for the multiple transmission time periods. The audio transmission data is divided accordingly based on the multiple transmission time periods to form multiple audio data segments, specifically:

[0075] Acquiring network transmission status information within the analysis period through real-time network transmission;

[0076] The network transmission status information includes delay, packet loss rate, bandwidth usage, and traffic statistics;

[0077] Based on the network transmission status information, a change assessment of the network status in an analysis period is performed, and multiple transmission time periods are divided according to the volatility of the network status;

[0078] A status level is set for each transmission time period, and each transmission time period is a continuous time period.

[0079] It should be noted that, the network conditions of an analysis period are evaluated for changes, and multiple transmission time periods are divided according to the volatility of the network status. Specifically, the division is based on the transmission quality and fluctuation changes of the network conditions. The multiple transmission time periods obtained have the characteristic of large differences in network conditions.

[0080] According to an embodiment of the present invention, the audio data segment is identified by a speech recognition model and user audio features are extracted from a system database, specifically:

[0081] Perform audio feature extraction and audio feature recognition on each audio data segment based on the speech recognition model, and mark the user information;

[0082] Based on user information, user data is retrieved and extracted from the system database to obtain user audio features.

[0083] It should be noted that the speech recognition model is based on deep learning and performs user speech recognition and corresponding interactive functions by learning audio features. The user audio features are the preferred feature data stored in the database and used for subsequent comparison with real-time audio data to analyze the credibility of the corresponding data.

[0084] According to an embodiment of the present invention, the audio data segment is subjected to audio feature extraction and feature vector conversion to form audio feature vector data, the audio feature vector data is reconstructed through an autoencoder network to generate reconstructed feature vector data, and the user audio feature and the reconstructed feature vector data are subjected to data similarity calculation to obtain audio feature similarity, specifically:

[0085] Extract audio features based on the audio data segment. The audio features include three dimensions: frequency, amplitude, and audio waveform. The extracted features are converted into feature vectors to obtain audio feature vector data.

[0086] Construct an autoencoder network, in which the audio feature vector data is used as high-dimensional input for data mapping through the MLP network, and mapped to low-dimensional representation data;

[0087] Through the decoder, the low-dimensional representation data is reconstructed to obtain high-dimensional representation data;

[0088] Use high-dimensional representation data as reconstructed feature vector data;

[0089] The difference between the reconstructed feature vector data and the audio feature vector data is judged by the mean square error loss function. If the difference is greater than the expected value, the autoencoder network parameters are updated, and data dimensionality reduction and data reconstruction are repeated until the expected reconstructed feature vector data is generated. The difference between the audio feature similarity at this time and the reconstructed feature vector data is recorded to obtain the audio feature similarity.

[0090] It should be noted that the MLP network stands for Multi-Layer Perceptron. The expected value is a user-set value. Determining differences is equivalent to determining similarities. Audio feature similarity reflects the difference between the low-dimensional representation of an audio data segment and the standard user audio features. The greater the similarity, the more accurate the data segment is for the subsequent simulated audio features.

[0091] According to an embodiment of the present invention, the data credibility is set according to the status level of each audio data segment and the audio feature similarity, the data credibility is used as the weight of each audio data segment, and data is extracted from the audio transmission data based on the weight. The extraction process uses the weight as the data ratio of each audio data segment to obtain the second audio transmission data, specifically:

[0092] Setting data credibility based on the status level of each audio data segment and the audio feature similarity, where the data credibility is equal to the weighted sum of the status level of the audio data segment and the audio feature similarity;

[0093] Using data credibility as the weight of each audio data segment, data extraction is performed from the audio transmission data. Specifically, the data extraction includes calculating the proportion of each audio data segment through the weight, and adjusting the data volume of each audio data segment based on the proportion until the data proportion corresponding to each audio data segment is equal to the proportion;

[0094] Finally, the second audio transmission data is obtained.

[0095] The calculation expression of data credibility R is as follows:

[0096] R=L×K1+S×K2;

[0097] L represents the status level, and S represents the audio feature similarity. K1 and K2 are preset weights. By changing the weights, the initially filtered second audio transmission data can be adjusted, freely setting the correlation between the status level of the corresponding data and the audio feature similarity.

[0098] It should be noted that the proportion is proportional to the weight of the audio data segment. The adjustment of the data volume of each audio data segment includes increasing and decreasing data. Reducing data means randomly deleting the data, and increasing data means randomly setting a certain segment in the audio data segment to copy to increase the data volume of the audio data segment.

[0099] According to an embodiment of the present invention, the GAN-based generative model is constructed, and the second audio transmission data is imported as real data into the generative model for feature learning and generating user audio training data, specifically:

[0100] Build a GAN-based generative model, which includes a generator and a discriminator;

[0101] Importing the second audio transmission data as real data into the generator for feature learning and simulation data generation;

[0102] Import the simulated data into the discriminator for recognition, and update the parameters of the generator and discriminator based on the preset loss function;

[0103] Conduct adversarial training based on the generator and discriminator cycles until the generator and discriminator reach Nash equilibrium;

[0104] A preset amount of simulated data is generated through the trained generative model and marked as user audio training data.

[0105] According to an embodiment of the present invention, the speech recognition model is trained using user audio training data, and user interaction recognition during the user network conference is performed using the speech recognition model, specifically:

[0106] Import user audio training data into the speech recognition model for model training;

[0107] During the real-time web conference data interaction process, user identification and data interaction are performed based on the speech recognition model.

[0108] According to an embodiment of the present invention, the further embodiment includes:

[0109] Through real-time network transmission, obtain the real-time audio transmission data of the user client in a real-time cycle;

[0110] Divide the real-time audio transmission data into multiple transmission time periods and corresponding multiple real-time data segments based on the network status;

[0111] Perform feature vector conversion on real-time data segments and reconstruct the data through the autoencoder network to obtain real-time reconstruction features;

[0112] Calculate the similarity between the real-time data segment and the real-time reconstructed feature data based on the standard Euclidean distance;

[0113] Analyze multiple real-time data segments to obtain multiple similarities;

[0114] The data compression rate is set based on the similarity. The higher the similarity, the higher the data compression rate. The audio compression algorithm is dynamically selected for each real-time data segment in the preset compression algorithm according to the data compression rate, and a compression algorithm transmission scheme is generated.

[0115] Based on the compression algorithm transmission solution, audio data compression transmission and data interaction are performed between the cloud platform and the user client.

[0116] It should be noted that the real-time period is a relatively short period of time. Each real-time data segment corresponds to a real-time reconstruction feature.

[0117] In the present invention, an autoencoding network is utilized to perform a dimensionality reduction process analysis on the real-time interactive audio data, and a similarity analysis is performed on the data before and after the output of the autoencoding network. The higher the similarity, the more important the corresponding audio feature is to the speech recognition model. Based on this, the compression rate is evaluated. The higher the similarity, the higher the compression rate, so as to ensure that the speech recognition model can effectively compress the audio data while maintaining a high recognition rate, reduce the amount of data transmission, and improve the real-time transmission efficiency of the audio data. A dynamic planning compression scheme is implemented, which further improves the real-time interaction capability between the cloud platform and the user client.

[0118] According to an embodiment of the present invention, dividing the multiple transmission time periods based on the network status further includes:

[0119] The complexity of the current network status is determined by the network transmission status information. If the complexity of the network status is higher than a predetermined level, multiple unit time periods are divided based on an analysis cycle.

[0120] Through network transmission status information, based on multiple dimensions such as delay, packet loss rate, bandwidth utilization, and traffic statistics, corresponding network feature data is generated for each unit time period.

[0121] Using K-means clustering, multiple network feature data are clustered and unit time periods are grouped to form K time groups;

[0122] The unit time periods in each time group are combined to generate a transmission time period, and K transmission time periods are generated based on the K time groups.

[0123] It's important to note that network complexity is a comprehensive assessment based on latency, packet loss rate, bandwidth utilization, and traffic statistics. Higher network complexity indicates a more unstable network. When dividing transmission time periods, network complexity can influence the time periods, allowing for clustering algorithms to identify highly classified time periods, enabling efficient data screening and extraction.

[0124] Figure 2 A block diagram of an intelligent speech recognition interaction system of the present invention is shown.

[0125] The second aspect of the present invention further provides an intelligent speech recognition interaction system 2, which includes: a memory 21 and a processor 22. The memory 21 includes an intelligent speech recognition interaction program. When the intelligent speech recognition interaction program is executed by the processor 22, the following steps are implemented:

[0126] Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the user client's audio transmission data within one analysis cycle;

[0127] Acquiring network transmission status information within the analysis period through real-time network transmission, dividing a plurality of transmission time periods based on the network status, setting status levels for the plurality of transmission time periods, and dividing the audio transmission data accordingly based on the plurality of transmission time periods to form a plurality of audio data segments;

[0128] Identify the audio data segment through the speech recognition model and extract the user audio features from the system database;

[0129] Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio features and the reconstructed feature vector data to obtain audio feature similarity;

[0130] Setting data credibility based on the status level of each audio data segment and the similarity between the audio features, using the data credibility as a weight for each audio data segment, and extracting data from the audio transmission data based on the weight, using the weight as a data ratio for each audio data segment in the extraction process, to obtain second audio transmission data;

[0131] Build a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data;

[0132] The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during user network conferences.

[0133] It is worth mentioning that in the real-time transmission and audio analysis of conferences, the conference system (or cloud platform) needs to perform real-time analysis and user identification on a large amount of audio data. This process requires fast and efficient recognition of audio data. Traditional audio interaction analysis technology is often based on simple audio recognition models with low recognition accuracy, resulting in poor classification of user audio data. There is also a lack of effective data mining and audio recognition model training and optimization processes for real-time audio data. There is also often a lack of corresponding training data, which leads to errors in the corresponding audio data recognition and system data entry, reducing the analysis efficiency of the audio analysis platform.

[0134] Based on this, the present invention divides the audio transmission data into multiple data segments, and through data dimensionality reduction and reconstruction analysis, it mines the important features in the audio data, and uses the audio feature similarity as an indicator. It further combines the network transmission characteristics to perform weight calculation, and determines the proportion of each data segment by the weight. It further extracts audio features with training significance from the audio transmission data, and subsequently simulates and generates them through the GAN model, so that better training data can be obtained from real-time transmission, and the speech recognition model can be further optimized.

[0135] Through the present invention, the audio recognition training effect and recognition effect based on the deep learning network are effectively improved, and the training data extraction process can be widely applied to audio platforms or cloud platforms in multiple scenarios, and has wide practicality.

[0136] According to an embodiment of the present invention, the cloud platform establishes a network connection with the user client and is applied to the user network conference process. Through real-time network transmission, the audio transmission data of the user client is obtained within an analysis cycle, specifically:

[0137] Based on the preset network transmission protocol, establish a network connection between the cloud platform and the user client;

[0138] Set an analysis cycle and collect data in one analysis cycle;

[0139] During the user's online conference, the audio transmission data of the user client is obtained through real-time network transmission and transmitted to the cloud platform.

[0140] It should be noted that the cloud platform is used to store, analyze, and identify the collected audio data, and further interact with users.

[0141] According to an embodiment of the present invention, the network transmission status information within the analysis period is obtained through real-time network transmission, multiple transmission time periods are divided based on the network status, and status levels are set for the multiple transmission time periods. The audio transmission data is divided accordingly based on the multiple transmission time periods to form multiple audio data segments, specifically:

[0142] Acquiring network transmission status information within the analysis period through real-time network transmission;

[0143] The network transmission status information includes delay, packet loss rate, bandwidth usage, and traffic statistics;

[0144] Based on the network transmission status information, a change assessment of the network status in an analysis period is performed, and multiple transmission time periods are divided according to the volatility of the network status;

[0145] A status level is set for each transmission time period, and each transmission time period is a continuous time period.

[0146] It should be noted that, the network conditions of an analysis period are evaluated for changes, and multiple transmission time periods are divided according to the volatility of the network status. Specifically, the division is based on the transmission quality and fluctuation changes of the network conditions. The multiple transmission time periods obtained have the characteristic of large differences in network conditions.

[0147] According to an embodiment of the present invention, the audio data segment is identified by a speech recognition model and user audio features are extracted from a system database, specifically:

[0148] Perform audio feature extraction and audio feature recognition on each audio data segment based on the speech recognition model, and mark the user information;

[0149] Based on user information, user data is retrieved and extracted from the system database to obtain user audio features.

[0150] It should be noted that the speech recognition model is based on deep learning and performs user speech recognition and corresponding interactive functions by learning audio features. The user audio features are the preferred feature data stored in the database and used for subsequent comparison with real-time audio data to analyze the credibility of the corresponding data.

[0151] According to an embodiment of the present invention, the audio data segment is subjected to audio feature extraction and feature vector conversion to form audio feature vector data, the audio feature vector data is reconstructed through an autoencoder network to generate reconstructed feature vector data, and the user audio feature and the reconstructed feature vector data are subjected to data similarity calculation to obtain audio feature similarity, specifically:

[0152] Extract audio features based on the audio data segment. The audio features include three dimensions: frequency, amplitude, and audio waveform. The extracted features are converted into feature vectors to obtain audio feature vector data.

[0153] Construct an autoencoder network, in which the audio feature vector data is used as high-dimensional input for data mapping through the MLP network, and mapped to low-dimensional representation data;

[0154] Through the decoder, the low-dimensional representation data is reconstructed to obtain high-dimensional representation data;

[0155] Use high-dimensional representation data as reconstructed feature vector data;

[0156] The difference between the reconstructed feature vector data and the audio feature vector data is judged by the mean square error loss function. If the difference is greater than the expected value, the autoencoder network parameters are updated, and data dimensionality reduction and data reconstruction are repeated until the expected reconstructed feature vector data is generated. The difference between the audio feature similarity at this time and the reconstructed feature vector data is recorded to obtain the audio feature similarity.

[0157] It should be noted that the MLP network stands for Multi-Layer Perceptron. The expected value is a user-set value. Determining differences is equivalent to determining similarities. Audio feature similarity reflects the difference between the low-dimensional representation of an audio data segment and the standard user audio features. The greater the similarity, the more accurate the data segment is for the subsequent simulated audio features.

[0158] According to an embodiment of the present invention, the data credibility is set according to the status level of each audio data segment and the audio feature similarity, the data credibility is used as the weight of each audio data segment, and data is extracted from the audio transmission data based on the weight. The extraction process uses the weight as the data ratio of each audio data segment to obtain the second audio transmission data, specifically:

[0159] Setting data credibility based on the status level of each audio data segment and the audio feature similarity, where the data credibility is equal to the weighted sum of the status level of the audio data segment and the audio feature similarity;

[0160] Using data credibility as the weight of each audio data segment, data extraction is performed from the audio transmission data. Specifically, the data extraction includes calculating the proportion of each audio data segment through the weight, and adjusting the data volume of each audio data segment based on the proportion until the data proportion corresponding to each audio data segment is equal to the proportion;

[0161] Finally, the second audio transmission data is obtained.

[0162] The calculation expression of data credibility R is as follows:

[0163] R=L×K1+S×K2;

[0164] L represents the status level, and S represents the audio feature similarity. K1 and K2 are preset weights. By changing the weights, the initially filtered second audio transmission data can be adjusted, freely setting the correlation between the status level of the corresponding data and the audio feature similarity.

[0165] It should be noted that the proportion is proportional to the weight of the audio data segment. The adjustment of the data volume of each audio data segment includes increasing and decreasing data. Reducing data means randomly deleting the data, and increasing data means randomly setting a certain segment in the audio data segment to copy to increase the data volume of the audio data segment.

[0166] According to an embodiment of the present invention, the GAN-based generative model is constructed, and the second audio transmission data is imported as real data into the generative model for feature learning and generating user audio training data, specifically:

[0167] Build a GAN-based generative model, which includes a generator and a discriminator;

[0168] Importing the second audio transmission data as real data into the generator for feature learning and simulation data generation;

[0169] Import the simulated data into the discriminator for recognition, and update the parameters of the generator and discriminator based on the preset loss function;

[0170] Conduct adversarial training based on the generator and discriminator cycles until the generator and discriminator reach Nash equilibrium;

[0171] A preset amount of simulated data is generated through the trained generative model and marked as user audio training data.

[0172] According to an embodiment of the present invention, the speech recognition model is trained using user audio training data, and user interaction recognition during the user network conference is performed using the speech recognition model, specifically:

[0173] Import user audio training data into the speech recognition model for model training;

[0174] During the real-time web conference data interaction process, user identification and data interaction are performed based on the speech recognition model.

[0175] The third aspect of the present invention also provides a computer-readable storage medium, which includes an intelligent voice recognition interaction program. When the intelligent voice recognition interaction program is executed by a processor, the steps of the intelligent voice recognition interaction method as described in any one of the above items are implemented.

[0176] The present invention discloses an intelligent speech recognition interaction method and system, which transmits user audio data and network status information in real time by establishing a network connection between a cloud platform and a user client. The transmission time period is divided and the level is set according to the network status, and the audio data is divided accordingly. The speech recognition model is used to identify audio features, and the feature vector is reconstructed through the autoencoding network to calculate the similarity of the audio features. The data credibility is set based on the status level and similarity and used as a weight to extract the second audio transmission data from the real-time audio. A GAN generation model is constructed and the second audio data is imported for feature learning to generate user audio training data for training the speech recognition model, effectively improving the audio recognition effect and improving the interaction efficiency of the speech recognition platform.

[0177] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0178] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0179] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0180] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0181] Alternatively, if the integrated units described above are implemented as software modules and sold or used as standalone products, they can also be stored on a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product, stored on a storage medium, includes instructions for enabling a computer device (such as a personal computer, server, or network device) to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as removable storage devices, ROM, RAM, magnetic disks, or optical disks.

[0182] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. An intelligent speech recognition interaction method, characterized in that: include: Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the user client's audio transmission data within one analysis cycle; Acquiring network transmission status information within the analysis period through real-time network transmission, dividing a plurality of transmission time periods based on the network status, setting status levels for the plurality of transmission time periods, and dividing the audio transmission data accordingly based on the plurality of transmission time periods to form a plurality of audio data segments; Identify the audio data segment through the speech recognition model and extract the user audio features from the system database; Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio features and the reconstructed feature vector data to obtain audio feature similarity; Setting data credibility based on the status level of each audio data segment and the similarity between the audio features, using the data credibility as a weight for each audio data segment, and extracting data from the audio transmission data based on the weight, using the weight as a data ratio for each audio data segment in the extraction process, to obtain second audio transmission data; Build a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data; The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during user network conferences.

2. The intelligent speech recognition interaction method according to claim 1, characterized in that: The cloud platform establishes a network connection with the user client and is applied to the user network conference process. Through real-time network transmission, the audio transmission data of the user client is obtained within an analysis cycle, specifically: Based on the preset network transmission protocol, establish a network connection between the cloud platform and the user client; Set an analysis cycle and collect data in one analysis cycle; During the user's online conference, the audio transmission data of the user client is obtained through real-time network transmission and transmitted to the cloud platform.

3. The intelligent speech recognition interaction method according to claim 2, characterized in that: The method comprises: obtaining network transmission status information within the analysis period through real-time network transmission, dividing a plurality of transmission time periods based on the network status, setting status levels for the plurality of transmission time periods, and performing corresponding data division on the audio transmission data based on the plurality of transmission time periods to form a plurality of audio data segments, specifically: Acquiring network transmission status information within the analysis period through real-time network transmission; The network transmission status information includes delay, packet loss rate, bandwidth usage, and traffic statistics; Based on the network transmission status information, a change assessment of the network status in an analysis period is performed, and multiple transmission time periods are divided according to the volatility of the network status; A status level is set for each transmission time period, and each transmission time period is a continuous time period.

4. The intelligent speech recognition interaction method according to claim 3, characterized in that: The audio data segment is identified by the speech recognition model and the user audio features are extracted from the system database, specifically: Perform audio feature extraction and audio feature recognition on each audio data segment based on the speech recognition model, and mark the user information; Based on user information, user data is retrieved and extracted from the system database to obtain user audio features.

5. The intelligent speech recognition interaction method according to claim 4, characterized in that: The audio data segment is subjected to audio feature extraction and feature vector conversion to form audio feature vector data, the audio feature vector data is reconstructed through an autoencoder network to generate reconstructed feature vector data, and the user audio feature and the reconstructed feature vector data are subjected to data similarity calculation to obtain audio feature similarity, specifically: Extract audio features based on the audio data segment. The audio features include three dimensions: frequency, amplitude, and audio waveform. The extracted features are converted into feature vectors to obtain audio feature vector data. Construct an autoencoder network, in which the audio feature vector data is used as high-dimensional input for data mapping through the MLP network, and mapped to low-dimensional representation data; Through the decoder, the low-dimensional representation data is reconstructed to obtain high-dimensional representation data; Use high-dimensional representation data as reconstructed feature vector data; The difference between the reconstructed feature vector data and the audio feature vector data is judged by the mean square error loss function. If the difference is greater than the expected value, the autoencoder network parameters are updated, and data dimensionality reduction and data reconstruction are repeated until the expected reconstructed feature vector data is generated. The difference between the audio feature similarity at this time and the reconstructed feature vector data is recorded to obtain the audio feature similarity.

6. The intelligent speech recognition interaction method according to claim 5, characterized in that: The data credibility is set according to the status level of each audio data segment and the audio feature similarity, the data credibility is used as the weight of each audio data segment, and data is extracted from the audio transmission data based on the weight. The extraction process uses the weight as the data ratio of each audio data segment to obtain the second audio transmission data, which is specifically: Setting data credibility based on the status level of each audio data segment and the audio feature similarity, where the data credibility is equal to the weighted sum of the status level of the audio data segment and the audio feature similarity; Using data credibility as the weight of each audio data segment, data extraction is performed from the audio transmission data. Specifically, the data extraction includes calculating the proportion of each audio data segment through the weight, and adjusting the data volume of each audio data segment based on the proportion until the data proportion corresponding to each audio data segment is equal to the proportion; Finally, the second audio transmission data is obtained.

7. The intelligent speech recognition interaction method according to claim 6, characterized in that: The GAN-based generative model is constructed, and the second audio transmission data is imported into the generative model as real data to perform feature learning and generate user audio training data, specifically: Build a GAN-based generative model, which includes a generator and a discriminator; Importing the second audio transmission data as real data into the generator for feature learning and simulation data generation; Import the simulated data into the discriminator for recognition, and update the parameters of the generator and discriminator based on the preset loss function; Conduct adversarial training based on the generator and discriminator cycles until the generator and discriminator reach Nash equilibrium; A preset amount of simulated data is generated through the trained generative model and marked as user audio training data.

8. The intelligent speech recognition interaction method according to claim 7, characterized in that: The speech recognition model is trained using user audio training data, and the speech recognition model is used to identify user interactions during the user network conference. Specifically, Import user audio training data into the speech recognition model for model training; During the real-time web conference data interaction process, user identification and data interaction are performed based on the speech recognition model.

9. An intelligent speech recognition interactive system, characterized in that: The system includes: a memory and a processor. The memory includes an intelligent speech recognition interaction program. When the intelligent speech recognition interaction program is executed by the processor, the following steps are implemented: Establish a network connection between the cloud platform and the user client, and apply it to the user's network conference process. Through real-time network transmission, obtain the user client's audio transmission data within one analysis cycle; Acquiring network transmission status information within the analysis period through real-time network transmission, dividing a plurality of transmission time periods based on the network status, setting status levels for the plurality of transmission time periods, and dividing the audio transmission data accordingly based on the plurality of transmission time periods to form a plurality of audio data segments; Identify the audio data segment through the speech recognition model and extract the user audio features from the system database; Performing audio feature extraction and feature vector conversion on the audio data segment to form audio feature vector data, reconstructing the audio feature vector data through an autoencoder network to generate reconstructed feature vector data, and calculating data similarity between the user audio features and the reconstructed feature vector data to obtain audio feature similarity; Setting data credibility based on the status level of each audio data segment and the similarity between the audio features, using the data credibility as a weight for each audio data segment, and extracting data from the audio transmission data based on the weight, using the weight as a data ratio for each audio data segment in the extraction process, to obtain second audio transmission data; Build a GAN-based generative model, import the second audio transmission data as real data into the generative model for feature learning and generate user audio training data; The speech recognition model is trained through user audio training data, and the speech recognition model is used to identify user interactions during user network conferences.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes an intelligent speech recognition interaction program. When the intelligent speech recognition interaction program is executed by the processor, the steps of the intelligent speech recognition interaction method according to any one of claims 1 to 8 are implemented.

Images