Speech recognition method, device and robot equipment based on artificial intelligence

Abstract

The present invention provides a speech recognition method, device and robot equipment based on artificial intelligence, the method comprising the following steps: obtaining a voice sample of a terminal user and extracting a sample baseline feature; when in a standby state, collecting a first area sound paragraph, preprocessing the first area sound paragraph, and extracting a target frequency domain feature; switching the working state according to the number of target frequency domain feature extractions; when in an explanation state, collecting a second area sound paragraph and performing a preprocessing step on the second area sound paragraph; extracting a specified speech vector feature from the preprocessed second area sound paragraph based on a specified frequency domain feature and using a speech recognition model, using a natural language processing technology to analyze the specified speech vector feature and generate a specified speech feedback text, and performing a speech explanation broadcast on the specified speech feedback text. The present invention has the effect of accurately identifying and processing user voice commands in a complex environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech recognition robots, and in particular relates to a speech recognition method, device and robot equipment based on artificial intelligence. Background Art

[0002] In scenarios such as visitor visits and public service guidance, it is usually necessary to arrange special personnel to provide explanations or guidance. When providing services, these manual guides often need to have rich knowledge reserves and good communication skills so that they can accurately convey information and answer various questions from visitors. However, manual explanations have the problems of low efficiency and lack of accuracy. On the one hand, manual guides have limited working time and energy, and it is difficult to maintain consistent service quality during long-term high-intensity work. On the other hand, due to differences in personal knowledge level and expression ability, the content of manual explanations may be inconsistent or wrong, affecting the visitor experience.

[0003] With the development of science and technology, robots with voice recognition capabilities are gradually replacing manual voice interactions in the above scenarios. These robots can provide continuous, stable and efficient explanation and guidance services through voice recognition technology and natural language processing capabilities. However, existing voice recognition robots usually rely on a single acoustic model and voice processing algorithm, which makes it impossible for existing voice recognition robots to effectively distinguish and process different voice signals when facing complex scenarios with multiple users and multiple environments. If the robot cannot quickly and accurately recognize and process the user's voice commands, it will greatly reduce the efficiency of the service and may cause user dissatisfaction. Therefore, how to improve the recognition accuracy and response speed of voice recognition robots in complex scenarios with multiple users and multiple environments has become an urgent problem to be solved in the current technological development. Summary of the invention

[0004] The present invention provides a speech recognition method, device and robot equipment based on artificial intelligence to solve the problem that the speech recognition robot cannot quickly and accurately recognize and process the user's voice instructions in complex scenarios.

[0005] In a first aspect, the present invention provides a speech recognition method based on artificial intelligence, which is applied to a robot device, wherein the working state of the robot device includes a standby state and a commentary state, and the speech recognition method based on artificial intelligence comprises the following steps:

[0006] Acquire voice samples of terminal users to which the mobile terminals belong through multiple different mobile terminals within the communication area where the robot device is located, and extract and store sample baseline features of all the voice samples using a voice recognition model;

[0007] When the working state is in the standby state, collecting a first area sound segment of the communication area within a preset time interval, performing a preprocessing step on the first area sound segment, and extracting a target frequency domain feature that matches any one or more of the sample benchmark features from the preprocessed first area sound segment;

[0008] If the number of extracted target frequency domain features is 0, the standby state is maintained, and the above-mentioned sound collection step and feature extraction step are repeated;

[0009] If the extraction quantity of the target frequency domain feature is 1, the target frequency domain feature is used as the designated frequency domain feature, and the working state is switched from the standby state to the explanation state;

[0010] If the number of extracted target frequency domain features is greater than 1, a target sound segment corresponding to each target frequency domain feature is separated from the preprocessed sound segment of the first region, a target time domain feature corresponding to the target frequency domain feature is obtained according to the target sound segment, a priority score corresponding to the target frequency domain feature is calculated using the target time domain feature, the target frequency domain feature with the highest priority score is used as the designated frequency domain feature, and the working state is switched from the standby state to the commentary state;

[0011] When the working state is in the explanation state, collecting a second area sound segment of the communication area and performing the preprocessing step on the second area sound segment;

[0012] Based on the designated frequency domain features and utilizing the speech recognition model, designated speech vector features are extracted from the preprocessed sound segment of the second region, natural language processing technology is used to analyze the designated speech vector features and generate designated speech feedback text, and the designated speech feedback text is voice-narrated and broadcasted.

[0013] Optionally, the step of obtaining voice samples of terminal users to which the mobile terminals belong through a plurality of different mobile terminals within the communication area where the robot device is located, and extracting and storing sample baseline features of all the voice samples using a voice recognition model comprises the following steps:

[0014] Establishing communication connections with all the mobile terminals based on connection requests sent by multiple different mobile terminals within the communication area where the robot device is located;

[0015] Acquire voice samples uploaded by all the mobile terminals, where the voice samples are a plurality of voice data pre-recorded on the mobile terminal by a terminal user holding the mobile terminal;

[0016] For each of the speech samples, normalize the speech sample, and perform a preprocessing step on the normalized speech sample;

[0017] Inputting the preprocessed speech sample into a preset speech recognition model, and outputting sample reference features of the speech sample in the frequency domain through the speech recognition model, wherein the speech recognition model includes a feature extraction layer, a convolution layer, a pooling layer, a fully connected layer, and a feature classification layer;

[0018] All the sample reference features are stored in a preset storage space.

[0019] Optionally, the preprocessing step specifically includes the following steps:

[0020] Taking the data to be preprocessed as the data to be processed, and constructing a time-domain step response function corresponding to the data to be processed;

[0021] Performing Laplace transform processing on the time-domain step response function to obtain an impulse response function corresponding to the data to be processed;

[0022] Extracting the data frequency domain features of the data to be processed based on the impulse response function and through Fourier transform;

[0023] Acquire the data energy spectrum value of the data to be processed by combining the data frequency domain features and a preset Teager operator;

[0024] The data energy spectrum value is filtered to obtain the data speech feature of the data to be processed.

[0025] Optionally, separating a target sound segment corresponding to each target frequency domain feature from the preprocessed sound segment of the first region, acquiring a target time domain feature corresponding to the target frequency domain feature according to the target sound segment, and calculating a priority score corresponding to the target frequency domain feature using the target time domain feature comprises the following steps:

[0026] For each of the target frequency domain features, calculating a feature matching value between the target frequency domain feature and each of the data voice features;

[0027] Performing inverse Fourier transform on the target data speech feature whose feature matching value exceeds a preset feature matching threshold to obtain the target data speech time domain feature;

[0028] Separating an initial target sound segment corresponding to the target frequency domain feature from the first region sound segment according to the target data speech time domain feature;

[0029] Using the target data speech time domain feature to modify the initial target sound segment to obtain the target sound segment corresponding to the target frequency domain feature;

[0030] A target time domain feature corresponding to the target frequency domain feature is acquired according to the target sound segment, and a priority score corresponding to the target frequency domain feature is calculated using the target time domain feature.

[0031] Optionally, the step of acquiring a target time domain feature corresponding to the target frequency domain feature according to the target sound segment, and calculating a priority score corresponding to the target frequency domain feature using the target time domain feature comprises the following steps:

[0032] Calculating the average amplitude of the target sound segment to obtain a first target time domain feature corresponding to the target frequency domain feature;

[0033] Counting the total time length of the target sound segment to obtain a second target time domain feature corresponding to the target frequency domain feature;

[0034] Calculating the time difference between the start time of the target sound segment and the start time of the first regional sound segment to obtain a third target time domain feature corresponding to the target frequency domain feature;

[0035] A score weighted calculation is performed in combination with the first target time domain feature, the second target time domain feature, and the third target time domain feature to obtain a priority score corresponding to the target frequency domain feature.

[0036] Optionally, extracting a target frequency domain feature matching any one or more sample reference features from the preprocessed first region sound segment comprises the following steps:

[0037] Using an independent component analysis method to separate a plurality of human voice features from the preprocessed sound segment in the first region;

[0038] Constructing a baseline probability density function of the sample baseline feature based on a multivariate Gaussian distribution, wherein the baseline probability density function includes a baseline weight parameter, a baseline mean vector, and a baseline covariance matrix;

[0039] For each of the human voice features, the Gaussian component likelihood of the human voice feature is calculated according to the reference probability density function, and the Gaussian component responsibility is calculated according to the Gaussian component likelihood;

[0040] Iteratively updating the benchmark weight parameter, the benchmark mean vector and the benchmark covariance matrix using the Gaussian component responsibility until the parameter update change of the benchmark weight parameter is less than a preset change threshold, and using the benchmark mean vector and the benchmark covariance matrix of the current iteration layer as the target mean vector and the target covariance matrix;

[0041] Obtaining a matching score between the human voice feature and the sample reference feature by calculating the target mean vector and the target covariance matrix;

[0042] If the matching score is greater than or equal to a preset score threshold, the human voice feature is used as a target frequency domain feature for matching the sample benchmark feature;

[0043] If the matching score is less than the score threshold, the human voice feature is discarded.

[0044] Optionally, the calculation formula of the Gaussian component likelihood is as follows:

[0045]

[0046] Where: x n ∈{x1,x2,...,x N} represents the nth human voice feature, represents the Gaussian component likelihood of the nth human voice feature calculated according to the baseline probability density function of the i-th sample baseline feature, where represents the base mean vector of the i-th base probability density function, represents the benchmark covariance matrix of the i-th benchmark probability density function, D represents the dimension of the multivariate Gaussian distribution, exp(·) represents the exponential function, and T represents the transposed matrix.

[0047] Optionally, the calculation formula of the Gaussian component responsibility is as follows:

[0048]

[0049] Where: γ i (x n ) represents the Gaussian component responsibility of the nth human voice feature calculated according to the benchmark probability density function of the i-th sample benchmark feature, represents the reference weight parameter of the i-th reference probability density function, represents the reference weight parameter of the j-th reference probability density function, It represents the Gaussian component likelihood of the nth human voice feature calculated according to the benchmark probability density function of the jth sample benchmark feature, and M represents the feature quantity of the sample benchmark feature.

[0050] In a second aspect, the present invention further provides an artificial intelligence-based speech recognition device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the artificial intelligence-based speech recognition method as described in the first aspect is implemented.

[0051] In a third aspect, the present invention further provides a robot device, comprising the artificial intelligence-based speech recognition device as described in the second aspect.

[0052] The beneficial effects of the present invention are:

[0053] By obtaining voice samples of multiple mobile terminal users in the communication area where the robot device is located, and using the voice recognition model to extract and store the baseline features of these samples, it is ensured that the voice recognition model can cover a wide range of user voice features. This method improves the robot device's recognition accuracy for different user voices and reduces the possibility of misrecognition. The robot device can dynamically switch its working state according to the collected sound segments. When the device is in standby mode, it determines whether it needs to switch to the interpretation state by extracting and matching the target frequency domain features. This dynamic switching mechanism ensures that the robot device can quickly enter the interpretation state when needed, improving the response speed and interpretation efficiency.

[0054] Through the preprocessing steps and feature matching methods, the target frequency domain features that match the sample baseline features are extracted from the sound segment, and the most appropriate features are selected as the designated frequency domain features based on the priority score. This method ensures the accuracy and relevance of the extracted target frequency domain features and improves the accuracy of subsequent speech recognition and interpretation. The preprocessing steps include time domain step response function, Laplace transform, Fourier transform and Teager operator methods to ensure the accuracy and effectiveness of the data frequency domain features extracted from the sound segment. The data speech features obtained through filtering can better reflect the actual speech features, reduce noise interference, and improve the accuracy of speech recognition.

[0055] By calculating the feature matching value between the target frequency domain features and the data voice features, and using the inverse Fourier transform and time domain feature correction method, the target sound segment is accurately extracted. The time domain features such as average amplitude, total time length and start time difference are combined for score weighted calculation to ensure the accuracy of the priority score, so as to select the most appropriate target frequency domain features. Multiple human voice features are separated by the independent component analysis method, and the benchmark probability density function of the sample benchmark features is constructed based on the multivariate Gaussian distribution. The benchmark parameters are iteratively updated using the Gaussian component likelihood and responsibility to ensure the accuracy of the matching score between the human voice features and the sample benchmark features. This method improves the accuracy of voice feature matching and reduces the probability of mismatching. In the commentary state, the robot device can extract the specified voice vector features based on the specified frequency domain features using the voice recognition model, and generate the specified voice feedback text through natural language processing technology. This method ensures the accuracy and coherence of the voice commentary content and improves the user experience and commentary effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flowchart of a speech recognition method based on artificial intelligence in one embodiment of the present application.

[0057] Figure 2 This is a schematic diagram of a process for obtaining a speech sample and extracting sample baseline features in one embodiment of the present application.

[0058] Figure 3 A schematic diagram of a process for calculating a priority score in one embodiment of the present application.

[0059] Figure 4 This is a schematic diagram of a process for extracting matching target frequency domain features in one implementation of the present application. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0061] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0062] Figure 1 FIG. 1 is a flow chart of a speech recognition method based on artificial intelligence in one embodiment. Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the above method may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps. The present invention discloses an artificial intelligence-based speech recognition method applied to a robot device, the working state of the robot device includes a standby state and a commentary state, such as Figure 1 As shown, the speech recognition method based on artificial intelligence disclosed in the present invention specifically includes the following steps:

[0063] S101. Acquire voice samples of terminal users of the mobile terminals through multiple different mobile terminals within the communication area where the robot device is located, and use a voice recognition model to extract and store sample baseline features of all voice samples.

[0064] Among them, the robot device can automatically detect and connect to all mobile terminals in the area, such as smart phones, tablets, etc. Through these mobile terminals, voice samples of terminal users are obtained. Voice samples can be obtained through natural voice interaction when users use mobile terminals. The obtained voice samples will be input into a pre-trained speech recognition model to extract features in the voice samples. This model can be based on deep learning technology, such as convolutional neural networks (CNN) and long short-term memory networks (LSTM). These features will be stored as sample baseline features, thereby establishing a voice feature database for mobile terminals corresponding to terminal users, thereby providing basic data support for subsequent speech recognition and processing.

[0065] S102. When the working state is in the standby state, collect the first area sound segment in the communication area within a preset time interval, perform a preprocessing step on the first area sound segment, and extract the target frequency domain features that match any one or more sample baseline features from the preprocessed first area sound segment.

[0066] Among them, when the working state is in the standby state, the first area sound segment of the communication area within the preset time interval is collected. In specific implementation, multiple microphone arrays can be set in the communication area, and these microphone arrays can collect surrounding sound data in real time. The preset time interval can be set according to the specific application requirements, for example, sound data is collected every 5 seconds. The collected sound data will be divided into multiple sound segments. Preprocessing steps are performed on these sound segments, including noise elimination, signal enhancement and spectrum analysis. Noise elimination can use adaptive filter technology, signal enhancement can be achieved through gain control, and spectrum analysis can be completed using fast Fourier transform (FFT). The preprocessed sound segment will be input into the speech recognition model to extract the target frequency domain features that match the sample benchmark features. The target frequency domain features refer to features that are similar to the sample benchmark features in the frequency domain, such as specific frequency peaks and spectrum shapes. Such processing effect is that useful speech features can be extracted from a complex sound environment, providing a reliable data basis for subsequent speech recognition and processing.

[0067] S103. Count the number of extracted target frequency domain features.

[0068] The number of target frequency domain features extracted from the preprocessed sound segment can be recorded by a counter. Every time the speech recognition model recognizes a target frequency domain feature, the value of the counter increases by 1. This counting process can continue until the entire sound segment is processed. The statistical result of the number of extractions will be used in the subsequent decision-making steps.

[0069] S104. If the number of extracted target frequency domain features is 0, the standby state is maintained, and the above-mentioned sound collection step and feature extraction step are repeated, that is, step S102 is repeated.

[0070] In the specific implementation, a threshold judgment condition can be set. When the value of the counter is 0, the system will maintain the current standby state and will not perform any further operations. At the same time, the system will restart the sound collection and preprocessing steps, that is, repeat step S102. The effect of such processing is that when no valid voice features are detected, the system can maintain a low power consumption state and continuously monitor the sound environment so as to respond in time when valid voice features are detected.

[0071] S105. If the number of extracted target frequency domain features is 1, the target frequency domain features are used as designated frequency domain features, and the working state is switched from the standby state to the explanation state.

[0072] Among them, in the specific implementation, when the value of the counter is 1, the system will record the currently extracted target frequency domain feature and set it as the specified frequency domain feature. Then, the system will change its working state from the standby state to the interpretation state. In the interpretation state, the system will be ready to perform subsequent voice processing and feedback tasks. The effect of such processing is that when the only valid voice feature is detected, it can quickly switch to the interpretation state and prepare for further voice interaction and processing.

[0073] S106. If the number of extracted target frequency domain features is greater than 1, the target sound segment corresponding to each target frequency domain feature is separated from the preprocessed first area sound segment, the target time domain feature corresponding to the target frequency domain feature is obtained according to the target sound segment, the priority score of the corresponding target frequency domain feature is calculated using the target time domain feature, the target frequency domain feature with the highest priority score is used as the designated frequency domain feature, and the working state is switched from the standby state to the commentary state.

[0074] Among them, in the specific implementation, the time window technology can be used to divide the sound segments, and the sound segments corresponding to each target frequency domain feature can be extracted separately. Then, the target time domain features corresponding to the target frequency domain features are obtained according to each target sound segment. The time domain features may include the amplitude and waveform of the sound. Using these time domain features, the priority score of each target frequency domain feature is calculated. The priority score can be calculated by a weighted average method, such as weighted summation of features such as frequency, amplitude and duration. The target frequency domain feature with the highest priority score will be set as the designated frequency domain feature, and the working state will be switched from standby to commentary. The effect of such processing is that when multiple valid speech features are detected, the most appropriate feature can be selected through priority evaluation to ensure that the system can accurately perform speech processing and feedback.

[0075] S107. When the working state is in the explanation state, collect the second area sound segment of the communication area and perform a preprocessing step on the second area sound segment.

[0076] The steps of collecting and preprocessing the sound segments in the second area are similar to those in the first area, including noise elimination, signal enhancement, and spectrum analysis. The preprocessed sound segments will be used for subsequent speech recognition and processing. The effect of such processing is that sound data can be collected and processed in real time during the commentary state, providing high-quality input for speech recognition and feedback.

[0077] S108. Based on the specified frequency domain features and using the speech recognition model, the specified speech vector features are extracted from the preprocessed second area sound segment, the specified speech vector features are analyzed using natural language processing technology and a specified speech feedback text is generated, and the specified speech feedback text is voice-narrated and broadcasted.

[0078] Among them, first, the preprocessed second area sound segment will be input into a high-precision speech recognition model. This model can adopt the most advanced deep learning technology, such as models based on Transformer architecture (such as BERT, GPT-3, etc.) or hybrid models (such as models combining CNN and RNN). These models can be pre-trained with a large amount of voice data, so as to have powerful voice feature extraction capabilities. After the model receives the preprocessed sound segment, it first needs to perform preliminary screening based on the specified frequency domain features, which are extracted and determined from the previous step. By matching these features, the model can quickly locate the part of the sound segment that may contain the target speech. This step can be achieved through spectral analysis technology, such as using fast Fourier transform (FFT) to convert the time domain signal into a frequency domain signal, and then performing feature matching in the frequency domain.

[0079] Next, the model will perform more in-depth feature extraction on the matched sound segments. Specifically, the model will extract the vector features of the speech, which include the frequency components, pitch changes, timbre characteristics, and timing information of the speech. These vector features can be extracted and represented by the multi-layer neural network in the model. In order to improve the accuracy of feature extraction, a multi-head attention mechanism can be used, which can capture the features of different levels and aspects in the speech signal, thereby generating more comprehensive and accurate speech vector features. After extracting the speech vector features, the next step is to use natural language processing (NLP) technology to analyze and understand these features. First, the speech vector features need to be converted into text representation, which can be achieved through speech-to-text technology. The speech-to-text conversion model can be based on the RNN or Transformer architecture and trained with a large amount of speech and text data, so as to have high-precision speech recognition capabilities.

[0080] After obtaining the text representation, NLP technology will perform semantic analysis and understanding on the text. Specifically, the text can be processed using pre-trained language models (such as BERT, GPT-3, etc.), which can understand the contextual semantics and grammatical structure of the text, thereby generating accurate semantic representations. Through semantic analysis, key information and intentions in the text can be identified, such as user inquiries, commands, or statements. Based on the results of the semantic analysis, the system will generate corresponding voice feedback text. This process can be achieved through natural language generation (NLG) technology. NLG technology can automatically generate natural and fluent text based on the semantic representation of the input. The generated text will contain the system's response to the user input, such as answering the user's questions, executing the user's commands, or providing relevant information.

[0081] Finally, the generated voice feedback text will be converted into a voice signal and broadcast. This process can be achieved through text-to-speech (TTS) technology. TTS technology can convert text into natural and fluent voice signals. The most advanced TTS technology, such as models based on WaveNet or Tacotron, can generate high-quality synthetic speech that is almost indistinguishable from human speech.

[0082] Through the above steps, the system can perform real-time speech recognition and feedback in the commentary state, providing an efficient voice interaction experience. In specific implementation, the above technical modules can be integrated into the system, including speech recognition model, speech-to-text conversion model, natural language processing model and text-to-speech conversion model. The various modules communicate and transfer data through standard data interfaces, thereby realizing the automation and efficiency of the entire speech processing process. Such a technical solution can not only improve the accuracy of speech recognition and processing, but also provide a natural and smooth voice interaction experience to meet the diverse needs of users.

[0083] In one implementation manner, step S101 specifically includes the following steps:

[0084] S201. Establish communication connections with all mobile terminals based on connection requests sent by multiple different mobile terminals within the communication area where the robot device is located.

[0085] Among them, first of all, it is necessary to ensure that the robot device has the ability to receive and process multiple connection requests. This is usually achieved through built-in wireless communication modules (such as Wi-Fi, Bluetooth, etc.). These modules can process connection requests from multiple devices at the same time, thereby establishing communication connections with these mobile terminals. In specific implementation, the communication module of the robot device will continuously scan the surrounding wireless signals. When the connection request of the mobile terminal is detected, it will establish a stable communication connection with each terminal through a handshake protocol (such as TCP / IP protocol). The handshake protocol process includes three handshakes: first, the mobile terminal sends a connection request (SYN packet) to the robot device, and the robot device replies with a confirmation signal (SYN-ACK packet) after receiving the request. Finally, the mobile terminal replies with a confirmation signal (ACK packet) to complete the connection establishment. This method ensures the reliability and stability of the communication connection. In addition, in order to improve the connection efficiency, parallel processing technology can be used, that is, multiple connection requests are processed simultaneously instead of one by one, thereby reducing the time to establish the connection. In this way, the robot device can establish communication connections with multiple mobile terminals in a short time, ensuring the smooth progress of subsequent steps.

[0086] S202. Acquire voice samples uploaded by all mobile terminals, where the voice samples are a plurality of voice data pre-recorded on the mobile terminal by the terminal user holding the mobile terminal.

[0087] Among them, the mobile terminal will record the user's voice through the built-in microphone and store the recorded voice data in the local storage. In order to upload these voice samples, the mobile terminal will transmit the voice data to the robot device through a wireless communication module (such as Wi-Fi, Bluetooth, etc.). During the transmission process, data compression technology (such as MP3, AAC, etc.) can be used to reduce the amount of transmitted data, thereby improving transmission efficiency. After receiving the voice sample, the robot device will verify the data to ensure that the data is not damaged or lost during the transmission process. The verification process is usually implemented through technologies such as checksum or cyclic redundancy check (CRC). These technologies can detect and correct errors in the transmission process, thereby ensuring the integrity and accuracy of the voice sample. In this way, the robot device can obtain high-quality voice samples and provide a reliable data basis for subsequent voice processing steps.

[0088] S203. For each speech sample, normalize the speech sample and perform a preprocessing step on the normalized speech sample.

[0089] Among them, the normalization process is to keep all samples in the same sample size and format for subsequent processing. In specific implementation, the normalization process includes volume normalization and time normalization. Volume normalization is to adjust the volume of all speech samples to the same level to avoid the influence of volume differences on subsequent processing. This can be achieved by calculating the volume mean and standard deviation of each sample, and then adjusting the sample volume to the preset standard volume. Time normalization is to adjust the length of all samples to the same time length, which can be achieved by truncation or padding. Truncation is to truncate the part exceeding the preset length, and padding is to fill the part less than the preset length with silence or repeated segments. The preprocessing steps include noise elimination, feature extraction, etc. Noise elimination is to remove background noise in the speech sample through a filter (such as a low-pass filter, a high-pass filter, etc.), thereby improving the clarity of the speech sample. Feature extraction is to provide input data for subsequent speech recognition by calculating the frequency components, pitch changes and other features of the speech sample. This can be achieved through technologies such as fast Fourier transform (FFT). Through normalization and preprocessing steps, the consistency and high quality of all speech samples can be ensured, providing a reliable data foundation for subsequent speech recognition.

[0090] S204. Input the preprocessed speech sample into a preset speech recognition model, and output the sample benchmark features of the speech sample in the frequency domain through the speech recognition model.

[0091] Among them, the speech recognition model includes a feature extraction layer, a convolution layer, a pooling layer, a fully connected layer, and a feature classification layer. The feature extraction layer is responsible for extracting preliminary frequency features from the preprocessed speech sample, which is usually achieved through the convolution operation in the convolutional neural network (CNN). The convolution layer performs a convolution operation on the input data through the convolution kernel to extract local features in the speech sample. The pooling layer reduces the size of the feature map through downsampling operations (such as maximum pooling, average pooling, etc.), thereby reducing the computational complexity and improving the generalization ability of the model. The fully connected layer flattens the feature map output by the pooling layer, and combines the local features into global features through a series of fully connected operations. The feature classification layer maps the global features to the preset classification label space through the softmax function, thereby outputting the frequency domain features of the speech sample. In specific implementation, pre-trained speech recognition models (such as DeepSpeech, Wav2Vec, etc.) can be used. These models are trained with a large amount of speech data and have powerful feature extraction and classification capabilities. In this way, the frequency domain features of the speech samples can be accurately extracted, providing basic data for subsequent storage and analysis.

[0092] S205. Store all sample benchmark features into a preset storage space.

[0093] Among them, the storage space can be a local memory (such as a hard disk, SSD, etc.) or a cloud storage service (such as AWS S3, Google Cloud Storage, etc.), and the specific choice depends on the application scenario and storage requirements. In order to improve storage efficiency, data compression techniques (such as Huffman coding, LZW compression, etc.) can be used to compress sample benchmark features, thereby reducing storage space usage. During the storage process, the integrity and security of the data need to be ensured. Integrity can be guaranteed by technologies such as checksum or cyclic redundancy check (CRC), which can detect and correct errors during the storage process. Security can be achieved through encryption technologies (such as AES, RSA, etc.), which can prevent unauthorized access and tampering. In addition, in order to facilitate subsequent data retrieval and analysis, metadata (such as sample ID, timestamp, user ID, etc.) can be added to each sample benchmark feature, which can be used as an index to quickly locate and retrieve specific sample benchmark features. In this way, a large number of speech sample benchmark features can be effectively managed and stored, providing reliable data support for subsequent analysis and application.

[0094] In one embodiment, the preprocessing step described in any one of the above embodiments specifically includes the following steps:

[0095] The data to be preprocessed is used as the data to be processed, and a time domain step response function corresponding to the data to be processed is constructed;

[0096] Performing Laplace transform processing on the time domain step response function to obtain the impulse response function corresponding to the data to be processed;

[0097] Extracting the frequency domain features of the data to be processed based on the impulse response function and Fourier transform;

[0098] Combining the frequency domain characteristics of the data with the preset Teager operator, the data energy spectrum value of the data to be processed is obtained;

[0099] The data energy spectrum value is filtered to obtain the data speech characteristics of the data to be processed.

[0100] In this embodiment, first, the data to be processed is usually a speech signal, which is expressed as a series of audio samples in the time domain. In order to perform preprocessing, the speech signal can be normalized first to ensure that the amplitude of the signal is within a standard range. The normalization process can be performed by calculating the maximum and minimum values ​​of the signal, and then adjusting each sample value of the signal to between 0 and 1. Next, a time domain step response function is constructed. The step response function refers to the response output by the system when a step signal (such as the Heaviside step function) is input. For a speech signal, it can be regarded as the output of a linear time-invariant system (LTI system). By processing the speech signal in segments, each segment can be approximated as the step response of an LTI system. The step response function can be calculated by the following formula:

[0101]

[0102] In the formula: α represents the amplitude, n represents the order, f i represents the center frequency of speech signal i, represents the initial phase value, b represents the attenuation factor, U(t i ) represents the unit step function, t i Describes the frequency of speech data i.

[0103] Then, the time-domain step response function needs to be Laplace transformed to obtain the impulse response function corresponding to the data to be processed. Laplace transform is a mathematical transformation that converts the time-domain signal to the complex frequency domain. It can convert the differential equation into an algebraic equation, thereby simplifying the analysis process. In specific implementation, the time-domain step response function g(t i ) to perform Laplace transform. The definition of Laplace transform is as follows:

[0104]

[0105] Where: s represents the complex frequency domain variable. By calculation, the step response function can be represented in the Laplace domain H(s). Next, by solving the inverse transform of the Laplace transform, the impulse response function can be obtained. The impulse response function is the response of the system to a unit impulse input (such as the Dirac delta function). In specific implementation, the properties of the Laplace transform can be used, that is, the relationship between the Laplace transform of the step response function and the Laplace transform of the impulse response function:

[0106]

[0107] Where: G i (z) represents the filter function, z represents the frequency after transformation, and j represents the impulse function.

[0108] Based on the impulse response function, the frequency domain features of the data to be processed are extracted through Fourier transform. Fourier transform is a mathematical transformation that converts time domain signals to frequency domain, which can decompose the signal into the superposition of different frequency components. In specific implementation, the impulse response function obtained in the previous step can be Fourier transformed. The definition of Fourier transform is as follows:

[0109]

[0110] By calculation, the representation of the impulse response function in the frequency domain G(f) can be obtained. Next, the frequency domain representation can be analyzed to extract the frequency domain features of the data to be processed. The frequency domain features include the amplitude spectrum and the phase spectrum. The amplitude spectrum represents the amplitude distribution of the signal at different frequencies, and the phase spectrum represents the phase distribution of the signal at different frequencies. In specific implementation, the amplitude spectrum and the phase spectrum can be obtained by calculating the modulus and phase angle of the frequency domain representation.

[0111] Next, we need to combine the frequency domain features of the data and the preset Teager operator to obtain the data energy spectrum value of the data to be processed. The Teager operator is a nonlinear operator that can effectively estimate the instantaneous energy of the signal and is particularly suitable for the processing of speech signals. In specific implementation, the energy spectrum value of the frequency domain features of the data can be calculated first. The energy spectrum value represents the energy distribution of the signal at different frequencies and can be obtained by calculating the square of the frequency domain features. The specific formula is as follows:

[0112] E(f)=|G(f)| 2

[0113] Next, the Teager operator is applied to process the energy spectrum value. The definition of the Teager operator is as follows:

[0114] Ψ[x(t)]=x(t) 2 -x(t-1)x(t+1)

[0115] Among them, x(t) is the time domain representation of the signal, and Ψ[x(t)] is the output of the Teager operator. For frequency domain features, they can be converted into time domain signals and then processed by the Teager operator. In specific implementation, the frequency domain features can be converted into time domain signals through inverse Fourier transform, and then the Teager operator can be applied to the time domain signal to obtain the energy spectrum value.

[0116] Finally, the data energy spectrum value needs to be filtered to obtain the data speech features of the data to be processed. Filtering refers to processing the signal through a filter to remove noise or extract specific frequency components. In specific implementation, you can select a suitable filter type (such as a low-pass filter, a high-pass filter, a band-pass filter, etc.) and filter parameters (such as cut-off frequency, filter order, etc.). Assuming that a band-pass filter is selected, the band-pass filter can remove other frequency components by retaining the signal components within a specific frequency range. By multiplying the energy spectrum value with the transfer function of the band-pass filter, the filtered energy spectrum value can be obtained. Then, the filtered energy spectrum value is converted back to the time domain signal through an inverse Fourier transform to obtain the speech features of the data. In this way, noise and irrelevant frequency components can be effectively removed, and the speech features of the data to be processed can be extracted. These speech features can be used in applications such as speech recognition and speech synthesis to provide high-quality input data.

[0117] In one embodiment, in step S106, a target sound segment corresponding to each target frequency domain feature is separated from the preprocessed first region sound segment, a target time domain feature corresponding to the target frequency domain feature is obtained according to the target sound segment, and a priority score corresponding to the target frequency domain feature is calculated using the target time domain feature, specifically comprising the following steps:

[0118] S301. For each target frequency domain feature, calculate the feature matching value between the target frequency domain feature and each data speech feature.

[0119] S302. Perform inverse Fourier transform on the target data speech features whose feature matching values ​​exceed the preset feature matching threshold to obtain the target data speech time domain features.

[0120] S303. Separate an initial target sound segment corresponding to the target frequency domain feature from the sound segment in the first region according to the target data speech time domain feature.

[0121] S304. Modify the initial target sound segment using the target data speech time domain features to obtain the target sound segment corresponding to the target frequency domain features.

[0122] S305. Obtain a target time domain feature corresponding to the target frequency domain feature according to the target sound segment, and calculate a priority score corresponding to the target frequency domain feature using the target time domain feature.

[0123] In this embodiment, first, the target frequency domain feature can be extracted from the target speech signal by Fourier transform, and represented as amplitude and phase information within a frequency range. Assume that the target frequency domain feature is T(f) and the data speech feature is D i (f), where i represents different data speech features. In order to calculate the feature matching value, common matching methods such as correlation coefficient method, Euclidean distance method or cosine similarity method can be used. Taking cosine similarity as an example, cosine similarity can measure the similarity between two vectors, and its definition is as follows:

[0124]

[0125] Among them, Sim(T, D i ) represents the cosine similarity between the target frequency domain feature and the i-th data voice feature. By calculating the cosine similarity between each data voice feature and the target frequency domain feature, a set of feature matching values ​​can be obtained. The higher the feature matching value, the greater the similarity between the two features. In this way, the matching degree between the target frequency domain feature and each data voice feature can be quantified, providing a basis for screening out the most relevant features in the subsequent steps.

[0126] Then, the target data speech features whose feature matching values ​​exceed the preset feature matching threshold need to be converted by inverse Fourier transform to obtain the target data speech time domain features. First, a feature matching threshold θ is set, and only the target data speech features whose feature matching values ​​are greater than θ will be selected. Assume that the feature matching value of the i-th data speech feature is Sim(T, D i ), and Sim(T, D i )>θ, then the feature is selected. Next, the selected data speech feature is subjected to inverse Fourier transform.

[0127] Next, we need to separate the initial target sound segment corresponding to the target frequency domain feature from the first region sound segment based on the target data speech time domain feature. First, the first region sound segment can be represented as a long speech signal, denoted as S(t). The goal is to extract the target data speech time domain feature d from this long signal. i (t) The initial target sound segment that matches. In specific implementation, the matching can be performed by sliding window technology. Set a window length L, slide the window in the first area sound segment, and calculate the matching degree between the signal in each window and the target data speech time domain feature. The matching degree can be measured by calculating the correlation coefficient between the two signals. The specific formula is as follows:

[0128]

[0129] Among them, Sw (t) represents the signal in the sliding window, and Respectively represent the mean of the time domain features of the signal and the target data voice in the sliding window. Through the sliding window technology, the window position with the highest matching degree can be found, and the signal in the window can be extracted as the initial target sound segment. In this way, the initial target sound segment corresponding to the target frequency domain features can be separated from the long-term voice signal, providing a basis for the subsequent correction steps.

[0130] Then, the initial target sound segment is corrected using the target data speech time domain features to obtain the target sound segment corresponding to the target frequency domain features. The purpose of the correction process is to ensure that the initial target sound segment more accurately reflects the target frequency domain features through further adjustment and optimization. In specific implementation, signal processing techniques such as time alignment, amplitude adjustment, and noise elimination can be used. First, time alignment is performed. Time alignment can be achieved through a cross-correlation function to find the optimal alignment position between the initial target sound segment and the target data speech time domain features. The specific formula is as follows:

[0131]

[0132] Among them, S init (t) is the initial target sound segment, τ is the time offset, and the optimal alignment position can be determined by finding the position of the maximum value. Next, amplitude adjustment is performed. Amplitude adjustment can be achieved by calculating the energy ratio of the initial target sound segment and the target data speech time domain characteristics. Finally, noise elimination is performed. Noise elimination can be achieved through filter technology such as low-pass filters or adaptive filters. By time alignment, amplitude adjustment and noise elimination of the initial target sound segment, a more accurate target sound segment can be obtained. In this way, the initial target sound segment can be further optimized to better reflect the target frequency domain characteristics.

[0133] Finally, the target time domain features corresponding to the target frequency domain features are obtained according to the target sound segment, and the priority scores of the corresponding target frequency domain features are calculated using the target time domain features. First, the target sound segment is converted to the frequency domain by Fourier transform to obtain an accurate representation of the target frequency domain features. Next, the corresponding target time domain features are extracted according to the target frequency domain features. The target time domain features can be obtained by inverse Fourier transform. By inverse Fourier transform, the target frequency domain features can be converted back to the time domain to obtain the target time domain features. Finally, the priority scores of the corresponding target frequency domain features are calculated using the target time domain features. The priority scores can be measured by indicators such as signal energy and correlation. In this way, the importance of the target frequency domain features can be quantified to obtain their priority scores. The higher the priority score, the greater the importance of the target frequency domain features in the target sound segment. In this way, a reference basis can be provided for subsequent speech processing and analysis.

[0134] In this implementation, step S305 specifically includes the following steps:

[0135] Calculate the average amplitude of the target sound segment to obtain a first target time domain feature corresponding to the target frequency domain feature;

[0136] Counting the total time length of the target sound segment to obtain a second target time domain feature corresponding to the target frequency domain feature;

[0137] Calculate the time difference between the start time of the target sound segment and the start time of the first region sound segment to obtain a third target time domain feature corresponding to the target frequency domain feature;

[0138] The first target time domain feature, the second target time domain feature and the third target time domain feature are combined to perform a score weighted calculation to obtain a priority score of the corresponding target frequency domain feature.

[0139] In this embodiment, when calculating the average amplitude of the target sound segment, it is first necessary to clarify the definition of the target sound segment. Assume that the target sound segment is y(t), which is represented as a signal in the time domain. The average amplitude is an important indicator to measure the signal strength, and is usually used to reflect the energy concentration of the signal. The steps for calculating the average amplitude are as follows: First, take the absolute value of the target sound segment to obtain the amplitude value at each time point. Then, sum these amplitude values ​​and divide them by the total number of time points to obtain the average amplitude. The specific formula is as follows:

[0140]

[0141] Among them, A avgrepresents the average amplitude of the target sound segment, T represents the total number of time points of the target sound segment, and y(t) represents the amplitude value of the target sound segment at time point t. In this way, the calculated average amplitude can reflect the overall intensity characteristics of the target sound segment. By calculating the average amplitude of the target sound segment, the first target time domain feature corresponding to the target frequency domain feature can be obtained, providing a basis for the subsequent priority score calculation.

[0142] When counting the total time length of the target sound segment, it is necessary to clarify the time range of the target sound segment. Assume that the target sound segment is y(t), its start time is t1, and its end time is t2. The total time length can be obtained by calculating the difference between the start time and the end time. The specific formula is L t =t2-t1. In this way, the total time length of the target sound segment can be accurately calculated. For example, if the target sound segment is a speech signal starting from 0.5 seconds and ending at 2.5 seconds, the start time is 0.5 seconds and the end time is 2.5 seconds. By calculating with the above formula, it can be obtained that the total time length of the speech signal is 2 seconds. By counting the total time length of the target sound segment, the second target time domain feature corresponding to the target frequency domain feature can be obtained, which provides a basis for the subsequent priority score calculation.

[0143] When calculating the time difference between the start time of the target sound segment and the start time of the first area sound paragraph, it is necessary to clarify the start time of the first area sound paragraph. Assume that the start time of the first area sound paragraph is t0, and the start time of the target sound segment is t1. The time difference can be obtained by calculating the difference between the start time of the target sound segment and the start time of the first area sound paragraph. The specific formula is Δt=t1-t0. By calculating the time difference between the start time of the target sound segment and the start time of the first area sound paragraph, the third target time domain feature corresponding to the target frequency domain feature can be obtained, which provides a basis for the subsequent priority score calculation.

[0144] The priority score can be obtained by P = δ1·A avg +δ2·L t +δ3·Δt, where P represents the priority score, the weight of the first target time domain feature is δ1, the weight of the second target time domain feature is δ2, and the weight of the third target time domain feature is δ3. In this way, the importance of different features can be comprehensively considered to obtain a comprehensive priority score. The higher the priority score, the greater the importance of the target frequency domain feature in the target sound segment.

[0145] In the above steps, the average amplitude is an important indicator to measure the strength of the audio signal. The larger the amplitude of an audio signal, the stronger the energy of the sound. By calculating the average amplitude, we can understand the overall strength of the sound. A higher average amplitude may indicate that the speaker's voice is clearer and more powerful, which helps to improve recognition accuracy. The total time length indicates the duration of the speaker's continuous speaking in a short period of time. It can also be understood as the duration of the voice command issued by the user. A longer total time length indicates that the same user may issue voice commands multiple times. The shorter the time difference, the faster the user issues voice commands in the same time period, and therefore the more priority should be given to respond. In summary, the priority score calculation based on the average amplitude, total time length, and time difference of the target sound segment can more accurately evaluate the importance and urgency of each voice command, thereby optimizing the processing order of voice commands and improving the overall response efficiency and user experience of the system.

[0146] In one embodiment, extracting the target frequency domain features matching any one or more sample reference features from the preprocessed first region sound segment in step S102 specifically includes the following steps:

[0147] S401. Separate multiple human voice features from the pre-processed first region sound segment using an independent component analysis method.

[0148] Among them, independent component analysis (ICA) is a statistical technique used to decompose multivariate signals into linear combinations of independent non-Gaussian signals. In this process, the preprocessed sound segment is used as input data, and it is decomposed into several independent components by the ICA method, each of which represents a human voice feature. In specific implementation, it is first necessary to center the preprocessed sound segment, that is, to subtract the mean of each sample so that the data mean is zero. Next, through whitening, the redundant information in the data is eliminated so that the covariance between different components is the unit matrix. Then, the ICA algorithm (such as FastICA) is applied for iterative optimization to find a set of independent components so that the statistical independence between these components is maximized. Finally, the independent components obtained are the separated human voice features. The effect of this process is that multiple independent human voice features can be effectively extracted from the mixed sound signal, providing a basis for subsequent feature matching and analysis.

[0149] S402. Construct a baseline probability density function of the sample baseline characteristics based on multivariate Gaussian distribution.

[0150] Among them, the benchmark probability density function includes a benchmark weight parameter, a benchmark mean vector and a benchmark covariance matrix. Multivariate Gaussian distribution is a common statistical model used to describe the distribution characteristics of multidimensional data. The benchmark probability density function is composed of a benchmark weight parameter, a benchmark mean vector and a benchmark covariance matrix. First, the benchmark mean vector represents the center position of the sample benchmark feature, and the benchmark covariance matrix describes the correlation and distribution range between the dimensions of the sample benchmark feature. By performing statistical analysis on the sample benchmark feature, its mean and covariance matrix are calculated. The benchmark weight parameter represents the contribution ratio of different Gaussian components in the mixture model. In specific implementation, the expectation maximization (EM) algorithm can be used to estimate these parameters. First, initialize the benchmark weight parameter, mean vector and covariance matrix, and then calculate the posterior probability of each sample belonging to each Gaussian component in the E step, and update the parameters according to these posterior probabilities in the M step, and iterate repeatedly until convergence. The final benchmark probability density function can accurately describe the distribution of the sample benchmark features and provide a basis for subsequent feature matching.

[0151] S403. For each human voice feature, the Gaussian component likelihood of the human voice feature is calculated according to the reference probability density function, and the Gaussian component responsibility is calculated according to the Gaussian component likelihood.

[0152] Among them, for each voice and speech feature, the Gaussian component likelihood of the voice and speech feature is calculated according to the reference probability density function, and the Gaussian component responsibility is calculated according to the Gaussian component likelihood. The Gaussian component likelihood represents the probability that the voice and speech feature belongs to a certain Gaussian component, and its calculation formula is:

[0153]

[0154] Where: x n ∈{x1,x2,...,x N} represents the nth voice feature, represents the Gaussian component likelihood of the nth human voice feature calculated based on the baseline probability density function of the i-th sample baseline feature, where represents the base mean vector of the i-th base probability density function, represents the baseline covariance matrix of the ith baseline probability density function, D represents the dimension of the multivariate Gaussian distribution, exp(·) represents the exponential function, and T represents the transposed matrix. Through the above formula, the likelihood of each Gaussian component of each voice and speech feature in the baseline probability density function can be calculated. Next, the Gaussian component responsibility is calculated based on the Gaussian component likelihood. The responsibility represents the probability that each voice and speech feature belongs to each Gaussian component, and its calculation formula is:

[0155]

[0156] Where: γ i (x n ) represents the Gaussian component responsibility of the nth human voice feature calculated based on the baseline probability density function of the i-th sample baseline feature, represents the base weight parameter of the i-th base probability density function, represents the base weight parameter of the jth base probability density function, It represents the Gaussian component likelihood of the nth human voice feature calculated according to the baseline probability density function of the jth sample baseline feature, and M represents the number of features of the sample baseline feature.

[0157] S404. Use Gaussian component responsibility to iteratively update the baseline weight parameters, baseline mean vector and baseline covariance matrix until the parameter update change of the baseline weight parameters is less than the preset change threshold, and use the baseline mean vector and baseline covariance matrix of the current iteration layer as the target mean vector and target covariance matrix.

[0158] Among them, by repeatedly iterating and updating the steps, the benchmark weight parameters, mean vector and covariance matrix are gradually adjusted to make the model better fit the data distribution. When the change of the benchmark weight parameter is less than the preset change threshold, the iteration process ends. Assuming that the number of iteration layers is K, the mean vector and covariance matrix at this time are the target mean vector and the target covariance matrix

[0159] S405. Obtain a matching score between the human voice features and the sample benchmark features by calculating the target mean vector and the target covariance matrix.

[0160] Among them, the matching score is used to measure the similarity between the human voice characteristics and the sample benchmark characteristics, and its calculation formula is:

[0161]

[0162] Through the above formula, the matching score between the human voice feature and the sample benchmark feature can be calculated. The higher the matching score, the higher the similarity between the two. In this way, the similarity between the human voice feature and the sample benchmark feature can be quantified, providing a basis for subsequent feature matching.

[0163] S406. If the matching score is greater than or equal to the preset score threshold, the human voice feature is used as the target frequency domain feature for sample benchmark feature matching.

[0164] Among them, the preset score threshold is used to screen the human voice features with higher matching scores. In specific implementation, a reasonable score threshold can be set according to the actual application scenario and data distribution. For each human voice feature, if its matching score is greater than or equal to the preset score threshold, it is considered to have a high similarity with the sample baseline feature, and it is saved and further processed as the target frequency domain feature.

[0165] S407. If the matching score is less than the score threshold, the human voice feature is discarded.

[0166] In the specific implementation, for each voice and speech feature, if its matching score is less than the preset score threshold, it will be removed from the feature set and no further processing will be performed. In this way, irrelevant or low-relevant voice and speech features can be effectively filtered out to ensure the accuracy and effectiveness of feature matching.

[0167] The present invention also discloses an artificial intelligence-based speech recognition device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the artificial intelligence-based speech recognition method described in any one of the above-mentioned embodiments is implemented.

[0168] Among them, the processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts within the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface diagrams and applications, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but implemented separately through a chip.

[0169] The memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory may also be optionally at least one storage device located away from the aforementioned processor.

[0170] The present invention also discloses a robot device, which may include the artificial intelligence-based speech recognition device described in the above embodiments, at least one network interface, a user interface, and a communication bus.

[0171] The communication bus is used to realize the connection and communication between these components.

[0172] The user interface may include a display interface and a camera interface, and the optional user interface may also include a standard wired interface and a wireless interface.

[0173] The network interface may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0174] In a robot device, the user interface is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor can be used to call an application program stored in the memory for a communication interference method for simulation training, and when executed by one or more processors, the robot device executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0175] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0176] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0177] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0178] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0179] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0180] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0181] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A speech recognition method based on artificial intelligence, characterized in that: Applied to a robot device, the working state of the robot device includes a standby state and an explanation state, and the method includes the following steps: Acquire voice samples of terminal users to which the mobile terminals belong through multiple different mobile terminals within the communication area where the robot device is located, and extract and store sample baseline features of all the voice samples using a voice recognition model; When the working state is in the standby state, collecting a first area sound segment of the communication area within a preset time interval, performing a preprocessing step on the first area sound segment, and extracting a target frequency domain feature that matches any one or more of the sample benchmark features from the preprocessed first area sound segment; If the number of extracted target frequency domain features is 0, the standby state is maintained, and the above-mentioned sound collection step and feature extraction step are repeated; If the extraction quantity of the target frequency domain feature is 1, the target frequency domain feature is used as the designated frequency domain feature, and the working state is switched from the standby state to the explanation state; If the number of extracted target frequency domain features is greater than 1, a target sound segment corresponding to each target frequency domain feature is separated from the preprocessed sound segment of the first region, a target time domain feature corresponding to the target frequency domain feature is obtained according to the target sound segment, a priority score corresponding to the target frequency domain feature is calculated using the target time domain feature, the target frequency domain feature with the highest priority score is used as the designated frequency domain feature, and the working state is switched from the standby state to the commentary state; When the working state is in the explanation state, collecting a second area sound segment of the communication area and performing the preprocessing step on the second area sound segment; Based on the designated frequency domain features and utilizing the speech recognition model, designated speech vector features are extracted from the preprocessed sound segment of the second region, natural language processing technology is used to analyze the designated speech vector features and generate designated speech feedback text, and the designated speech feedback text is voice-narrated and broadcasted.

2. The artificial intelligence-based speech recognition method according to claim 1, characterized in that: The method of obtaining voice samples of terminal users to which the mobile terminals belong through multiple different mobile terminals within the communication area where the robot device is located, and extracting and storing sample baseline features of all the voice samples using a voice recognition model comprises the following steps: Establishing communication connections with all the mobile terminals based on connection requests sent by multiple different mobile terminals within the communication area where the robot device is located; Acquire voice samples uploaded by all the mobile terminals, where the voice samples are a plurality of voice data pre-recorded on the mobile terminal by a terminal user holding the mobile terminal; For each of the speech samples, normalize the speech sample, and perform a preprocessing step on the normalized speech sample; Inputting the preprocessed speech sample into a preset speech recognition model, and outputting sample reference features of the speech sample in the frequency domain through the speech recognition model, wherein the speech recognition model includes a feature extraction layer, a convolution layer, a pooling layer, a fully connected layer, and a feature classification layer; All the sample reference features are stored in a preset storage space.

3. The artificial intelligence-based speech recognition method according to claim 1 or 2, characterized in that: The pre-processing step specifically comprises the following steps: Taking the data to be preprocessed as the data to be processed, and constructing a time-domain step response function corresponding to the data to be processed; Performing Laplace transform processing on the time-domain step response function to obtain an impulse response function corresponding to the data to be processed; Extracting the data frequency domain features of the data to be processed based on the impulse response function and through Fourier transform; Acquire the data energy spectrum value of the data to be processed by combining the data frequency domain features and a preset Teager operator; The data energy spectrum value is filtered to obtain the data speech feature of the data to be processed.

4. The artificial intelligence-based speech recognition method according to claim 3, characterized in that: The step of separating a target sound segment corresponding to each target frequency domain feature from the preprocessed sound segment of the first region, obtaining a target time domain feature corresponding to the target frequency domain feature according to the target sound segment, and calculating a priority score corresponding to the target frequency domain feature using the target time domain feature comprises the following steps: For each of the target frequency domain features, calculating a feature matching value between the target frequency domain feature and each of the data voice features; Performing inverse Fourier transform on the target data speech feature whose feature matching value exceeds a preset feature matching threshold to obtain the target data speech time domain feature; Separating an initial target sound segment corresponding to the target frequency domain feature from the first region sound segment according to the target data speech time domain feature; Using the target data speech time domain feature to modify the initial target sound segment to obtain a target sound segment corresponding to the target frequency domain feature; A target time domain feature corresponding to the target frequency domain feature is acquired according to the target sound segment, and a priority score corresponding to the target frequency domain feature is calculated using the target time domain feature.

5. The artificial intelligence-based speech recognition method according to claim 4, characterized in that: The step of obtaining a target time domain feature corresponding to the target frequency domain feature according to the target sound segment, and calculating a priority score corresponding to the target frequency domain feature using the target time domain feature comprises the following steps: Calculating the average amplitude of the target sound segment to obtain a first target time domain feature corresponding to the target frequency domain feature; Counting the total time length of the target sound segment to obtain a second target time domain feature corresponding to the target frequency domain feature; Calculating the time difference between the start time of the target sound segment and the start time of the first regional sound segment to obtain a third target time domain feature corresponding to the target frequency domain feature; A score weighted calculation is performed in combination with the first target time domain feature, the second target time domain feature, and the third target time domain feature to obtain a priority score corresponding to the target frequency domain feature.

6. The artificial intelligence-based speech recognition method according to claim 1, characterized in that: The step of extracting a target frequency domain feature that matches any one or more sample reference features from the preprocessed first region sound segment comprises the following steps: Using an independent component analysis method to separate a plurality of human voice features from the preprocessed sound segment in the first region; Constructing a baseline probability density function of the sample baseline feature based on a multivariate Gaussian distribution, wherein the baseline probability density function includes a baseline weight parameter, a baseline mean vector, and a baseline covariance matrix; For each of the human voice features, the Gaussian component likelihood of the human voice feature is calculated according to the reference probability density function, and the Gaussian component responsibility is calculated according to the Gaussian component likelihood; Iteratively updating the benchmark weight parameter, the benchmark mean vector and the benchmark covariance matrix using the Gaussian component responsibility until the parameter update change of the benchmark weight parameter is less than a preset change threshold, and using the benchmark mean vector and the benchmark covariance matrix of the current iteration layer as the target mean vector and the target covariance matrix; Obtaining a matching score between the human voice feature and the sample reference feature by calculating the target mean vector and the target covariance matrix; If the matching score is greater than or equal to a preset score threshold, the human voice feature is used as a target frequency domain feature for matching the sample benchmark feature; If the matching score is less than the score threshold, the human voice feature is discarded.

7. The artificial intelligence-based speech recognition method according to claim 6, characterized in that: The calculation formula of the Gaussian component likelihood is as follows: Where: x n ∈{x1,x2,...,x N } represents the nth human voice feature, represents the Gaussian component likelihood of the nth human voice feature calculated according to the baseline probability density function of the i-th sample baseline feature, where represents the base mean vector of the i-th base probability density function, represents the benchmark covariance matrix of the i-th benchmark probability density function, D represents the dimension of the multivariate Gaussian distribution, exp(·) represents the exponential function, and T represents the transposed matrix.

8. The artificial intelligence-based speech recognition method according to claim 7, characterized in that: The calculation formula of the Gaussian component responsibility is as follows: Where: γ i (x n ) represents the Gaussian component responsibility of the nth human voice feature calculated according to the benchmark probability density function of the i-th sample benchmark feature, represents the reference weight parameter of the i-th reference probability density function, represents the reference weight parameter of the j-th reference probability density function, It represents the Gaussian component likelihood of the nth human voice feature calculated according to the benchmark probability density function of the jth sample benchmark feature, and M represents the feature quantity of the sample benchmark feature.

9. A speech recognition device based on artificial intelligence, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the artificial intelligence-based speech recognition method as described in any one of claims 1 to 8 is implemented.

10. A robot device, characterized in that: Including the artificial intelligence based speech recognition device as described in claim 9.

Images