Voice control method, system and equipment for machine room inspection robot and storage medium

Abstract

The invention relates to the technical field of voice recognition, and particularly provides a voice control method, system and device for a machine room inspection robot and a storage medium, and the method comprises the steps: receiving a voice signal, and extracting a voice feature from the voice signal; identifying accent information and semantic information of the voice features by using a first model; confirming that the semantic information contains an inspection related instruction, and the inspection related instruction has an authentication requirement; inputting the voice features and the accent information into a second model to obtain identity information to which the voice signal belongs; acquiring a permission corresponding to the identity information, confirming that the permission meets the authentication requirement, and executing the routing inspection related instruction; according to the method, the authority management of the inspection robot from the voice dimension is realized, and the feature recognition is performed from the accent dimension and the semantic dimension by combining the attention mechanism, the long-short term memory network and the support vector machine, so that the accuracy of voice recognition is improved, and the method can adapt to the high-noise interference environment of a machine room.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech recognition, and in particular relates to a speech control method, system, equipment and storage medium of a machine room inspection robot. Background Art

[0002] In the operation and maintenance management of data centers, computer room inspection robots play a key role. They mainly carry out comprehensive inspections of data centers based on pre-set patrol routes. However, when there is a need to temporarily adjust the patrol route or change the patrol content, managers must manually modify the patrol parameters. This operation process is not only complicated, but also very cumbersome, which greatly affects the efficiency of emergency response.

[0003] With the rapid development of science and technology, voice interaction technology has emerged and rapidly risen, bringing new hope for solving the problem of human-computer interaction. Through voice interaction, the difficulty of interaction between people and devices can be significantly reduced, making operation more convenient and efficient. However, in the computer room environment, there are differences in personnel permissions. Accurately judging whether the person issuing the command has the corresponding operation permission has become an important issue that needs to be solved in the application of voice interaction.

[0004] At present, common authentication methods, such as token swiping, have many drawbacks. On the one hand, this method requires additional hardware equipment on the cruise robot, which increases the cost and system complexity; on the other hand, personnel must walk close to the cruise robot to perform authentication operations, which is extremely inconvenient in practical applications and also limits the flexibility and timeliness of authentication. In addition, the noise interference in the computer room environment is relatively large, and how to improve the accuracy of voice recognition is also a technical problem that needs to be solved. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a voice control method, system, device and storage medium for a computer room inspection robot to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a voice control method for a machine room inspection robot, comprising: receiving a speech signal, and extracting speech features from the speech signal; recognizing the accent information and semantic information of the speech feature using the first model; Confirming that the semantic information includes inspection-related instructions, and the inspection-related instructions have authentication requirements; Inputting the speech feature and the accent information into a second model to obtain identity information to which the speech signal belongs; Obtain the authority corresponding to the identity information, confirm that the authority meets the authentication requirements, and execute the inspection-related instructions; The first model includes an attention mechanism and a long short-term memory network; The second model includes a feature fusion layer and a support vector machine.

[0007] In an optional implementation, a speech signal is received, and speech features are extracted from the speech signal, wherein the speech features include phoneme features and prosodic features, including: Convert the speech signal from the time domain to the frequency domain, obtain spectrum information by transforming the spectrum, wherein the transforming process includes framing, windowing, and fast Fourier transform; map the spectrum information to the Mel frequency scale, and then extract the Mel frequency cepstrum coefficients by discrete cosine transform operation; Performing linear prediction sharing on the speech signal, estimating vocal tract model parameters of the speech, and converting the vocal tract model parameters into linear prediction cepstrum coefficients; Extracting a phoneme sequence of a speech signal using a hidden Markov model, determining the pronunciation characteristics of the phoneme by analyzing the acoustic pattern of each phoneme in the phoneme sequence, and counting the occurrence frequency and distribution data of the phoneme in the phoneme sequence; integrating the phoneme sequence, the pronunciation characteristics, the occurrence frequency and the distribution data of each phoneme into a phoneme feature sequence; The intonation information, stress information and rhythm information of the speech signal are extracted and integrated into prosodic features.

[0008] In an optional implementation, receiving a speech signal and extracting speech features from the speech signal includes: Preprocessing the speech signal, wherein the preprocessing includes denoising; Speech features are extracted from the preprocessed speech signal, and the speech features are normalized.

[0009] In an optional implementation, preprocessing the speech signal includes: Add random Gaussian white noise of equal length to the time series of speech signal multiple times; After each noise is added, the speech signal after the noise is added is decomposed into a plurality of intrinsic mode function components and a residual component by using the empirical mode decomposition technique to obtain a plurality of groups of decomposed data, each group of the decomposed data including a plurality of intrinsic mode function components and a residual component; The average values ​​of multiple intrinsic mode function components in multiple groups of decomposed data are calculated respectively to obtain multiple signal components; Sorting the plurality of signal components according to a rule from high frequency to low frequency; Calculating the energies of a plurality of the signal components and normalizing the energies into relative energies; determining an energy entropy of each of said signal components based on said relative energies; The signal component with the largest energy entropy is marked as the target component that dominates the noise; Perform wavelet transform on the speech signal based on the preset number of decomposition layers, and calculate the length L of the wavelet coefficients of each layer; Extract the noise variance of the target component , and based on the noise variance Calculate the decomposition threshold, the calculation formula is:

[0010] Where i represents the i-th layer of wavelet transform; The decomposition threshold is used to construct a noise reduction function, wherein the noise reduction function comprises:

[0011] in, is the jth coefficient of the i-th layer wavelet decomposition component after denoising, is the jth coefficient of the i-th layer wavelet decomposition component, is the adjustment parameter, n is the parameter that determines the shape of the function, and sgn() is the sign function; Using the noise reduction function to perform noise reduction processing on the signal component arranged before the target component to obtain a noise-reduced high-frequency component; The noise reduction high frequency component and the signal component arranged after the target component are reconstructed to obtain a noise reduction speech signal.

[0012] In an optional embodiment, the method further comprises: Generating a timestamp for the voice feature, and caching the voice feature with the timestamp in a designated area; Marking the timestamps of the speech features to which the accent information and the semantic information belong; Check whether the timestamps of the speech features and accent information input into the second model are consistent, and if they are inconsistent, generate an error message.

[0013] In an optional embodiment, the first model includes: A feature input layer, wherein the input dimension of the feature input layer is 128 dimensions, and is used to perform normalization and dimensionality reduction processing on the input speech features, thereby reducing the 128-dimensional speech features to 64 dimensions; Attention mechanism layer, including 64-dimensional hidden layer, using scaled dot product attention; The long short-term memory network has 256 hidden units and 2 layers. The forget gate, input gate and output gate use the Sigmoid function as the activation function. The long short-term memory network adopts a multi-task learning mechanism, and the output layer includes a first branch and a second branch, the first branch is used to output accent information, and the second branch is used to output semantic information.

[0014] In an optional embodiment, the second model includes: The feature fusion layer is used to perform weighted summation of input features to obtain fused features; A support vector machine, used to output identity information corresponding to the fusion feature, the support vector machine including a kernel function, a kernel coefficient and a penalty factor; The kernel function is used to map the input features to a high-dimensional space to find an optimal classification hyperplane in the high-dimensional space, and the kernel function is a polynomial kernel function.

[0015] In a second aspect, the present invention provides a voice control system for a machine room inspection robot, comprising: A feature extraction module, used for receiving a speech signal and extracting speech features from the speech signal; A first recognition module, used for recognizing the accent information and semantic information of the speech feature by using a first model; A first processing module, used for confirming that the semantic information includes inspection-related instructions, and the inspection-related instructions have authentication requirements; A second recognition module, used for inputting the voice feature and the accent information into a second model to obtain identity information to which the voice signal belongs; A second processing module is used to obtain the authority corresponding to the identity information, confirm that the authority meets the authentication requirements, and execute the inspection-related instructions; The first model includes an attention mechanism and a long short-term memory network; The second model includes a feature fusion layer and a support vector machine.

[0016] In a third aspect, a device is provided, including: A memory, used to store the voice control program of the computer room inspection robot; The processor is used to implement the steps of the voice control method of the computer room inspection robot provided in the first aspect when executing the voice control program of the computer room inspection robot.

[0017] In a fourth aspect, a computer-readable storage medium is provided, on which a voice control program for a machine room inspection robot is stored. When the voice control program for the machine room inspection robot is executed by a processor, the steps of the voice control method for the machine room inspection robot provided in the first aspect are implemented.

[0018] The beneficial effect of the present invention is that the voice control method, system, device and storage medium of the computer room inspection robot provided by the present invention recognizes the accent and semantics of the voice signal, and confirms the identity by combining the accent and voice signal, and then determines whether to execute the instructions contained in the semantic information based on the authority information corresponding to the identity information. In this way, the authority management of the inspection robot from the voice dimension is realized, and by combining the attention mechanism, long short-term memory network and support vector machine, feature recognition is performed from the two dimensions of accent and semantics, which improves the accuracy of voice recognition and can adapt to the high noise interference environment of the computer room.

[0019] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0022] Figure 2 It is a schematic flow chart of a voice interaction scenario of a method according to an embodiment of the present invention.

[0023] Figure 3 It is a schematic flow chart of a voice instruction execution scenario of a method according to an embodiment of the present invention.

[0024] Figure 4 is a schematic block diagram of a system according to an embodiment of the present invention.

[0025] Figure 5 A schematic diagram of the structure of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0028] The voice control method of the computer room inspection robot provided in the embodiment of the present invention is executed by a computer device, and accordingly, the voice control system of the computer room inspection robot runs in the computer device.

[0029] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject may be a voice control system of a machine room inspection robot. According to different requirements, the order of the steps in the flow chart may be changed, and some may be omitted.

[0030] like Figure 1 As shown, the method includes: S1. Receive a speech signal and extract speech features from the speech signal.

[0031] First, voice signals are received through devices such as microphone arrays. These voice signals appear as continuous analog signals in the time domain. In order to facilitate subsequent digital signal processing, they need to be converted into digital signals. Then, classic algorithms such as Mel-frequency cepstral coefficients (MFCC) or linear prediction cepstral coefficients (LPCC) are used to extract voice features from the digitized voice signals. These voice features can effectively characterize the essential properties of voice signals and provide a key data basis for subsequent voice analysis.

[0032] S2. Using the first model to identify the accent information and semantic information of the speech feature.

[0033] The first model is used to conduct in-depth analysis of the extracted speech features. This first model combines the Attention Mechanism and the Long Short-Term Memory (LSTM) network. The Attention Mechanism enables the model to automatically focus on key parts when processing speech features, enhance the focus on important information, and thus improve the accuracy of recognition. The Long Short-Term Memory Network is good at processing time series data, and can effectively capture the long-term dependencies and short-term dynamic changes in speech, and accurately identify the accent information and semantic information contained in the speech features. Accent information helps to determine the speaker's regional background, while semantic information is the core of understanding the content of speech.

[0034] S3. Confirm that the semantic information includes inspection-related instructions, and the inspection-related instructions have authentication requirements.

[0035] After obtaining the semantic information, the system will analyze it in depth to determine whether the semantic information contains inspection-related instructions. Inspection-related instructions are usually specific instructions for professional scenarios such as equipment status monitoring and environmental parameter detection. If it is confirmed that such instructions are included, further check whether the inspection-related instructions have authentication requirements. The authentication requirement is to ensure that only personnel with corresponding permissions can execute specific instructions to ensure the security and stability of the system.

[0036] S4. Input the speech feature and the accent information into a second model to obtain the identity information to which the speech signal belongs.

[0037] The extracted speech features and the recognized accent information are input into the second model. The second model consists of a feature fusion layer and a support vector machine (SVM). The feature fusion layer organically combines the speech features and accent information to generate a comprehensive feature vector that contains more comprehensive personal identity-related information. Based on this comprehensive feature vector, the support vector machine uses its powerful classification capabilities to accurately determine the identity information of the speech signal and effectively identify the speaker's identity.

[0038] S5. Obtain the authority corresponding to the identity information, confirm that the authority meets the authentication requirements, and execute the inspection-related instructions.

[0039] After obtaining the identity information, the system will immediately query the permissions corresponding to the identity information. By comparing the permissions database, it is confirmed whether the permissions meet the authentication requirements of the inspection-related instructions. If the permissions meet the requirements, the system will smoothly execute the inspection-related instructions and complete a series of predetermined inspection tasks such as equipment status query and fault diagnosis to ensure the normal operation and timely maintenance of the system. If the permissions are not met, the instructions will be refused to execute and the corresponding prompt information will be given to ensure the security access control of the system.

[0040] In an embodiment of the present invention, based on step S1, an example is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0041] S101. Voice signal noise reduction In order to cope with the noise interference in the computer room environment, a noise reduction method is provided, comprising the following steps: (1) Add random Gaussian white noise of equal length to the time series of the speech signal multiple times.

[0042] In the time series processing of speech signals, in order to fully simulate the noise interference in the real environment, we will add random Gaussian white noise of equal length to the speech signal many times. Gaussian white noise is a kind of noise with normal distribution characteristics. Its power spectrum density is evenly distributed in the entire frequency range, which can well simulate various complex background noises. The length of each added noise is carefully set to ensure that the noise interference is fully introduced without changing the main characteristics of the speech signal.

[0043] A special random number generation algorithm is used, combined with the probability density function of the Gaussian distribution, to generate a noise sequence that conforms to the characteristics of Gaussian white noise, and then superimpose it on the time series of the speech signal. By repeating this operation multiple times, multiple speech signal samples with different noise interference levels can be obtained, providing a rich data basis for subsequent analysis.

[0044] (2) Each time after noise is added, the speech signal after the noise is added is decomposed into a plurality of intrinsic mode function components and a residual component using the empirical mode decomposition technique, thereby obtaining a plurality of groups of decomposition data, each group of the decomposition data including a plurality of intrinsic mode function components and a residual component.

[0045] After each noise addition, the Empirical Mode Decomposition (EMD) technique is used to process the speech signal after the noise is added. EMD is an adaptive signal processing method that can decompose complex nonlinear and non-stationary signals into a series of intrinsic mode function (IMF) components with different characteristic scales and a residual component.

[0046] The decomposition process is based on the local characteristic time scale of the signal. Through continuous screening and iteration, the components of different frequencies in the signal are separated step by step. After EMD processing, each speech signal with added noise can obtain multiple sets of decomposition data, each set of decomposition data contains multiple IMF components and a residual component. These IMF components represent the fluctuation characteristics of the signal on different time scales, and the residual component reflects the overall trend of the signal.

[0047] (3) Calculate the average values ​​of multiple intrinsic mode function components in multiple groups of decomposed data respectively to obtain multiple signal components.

[0048] For each of the multiple groups of decomposed data, the average values ​​of the multiple intrinsic mode function components are calculated. The purpose of this step is to eliminate the local fluctuations caused by the randomness of noise addition in each group of decomposed data and extract more representative signal features.

[0049] For each IMF component, traverse all the components obtained by decomposing the speech signal after adding noise, add up the values ​​of these components, and then divide them by the number of samples to get the average value of the IMF component. In this way, multiple signal components are obtained, which can more stably reflect the characteristics of the speech signal at different frequency scales.

[0050] (4) Sort the plurality of signal components in order from high frequency to low frequency.

[0051] The obtained multiple signal components are sorted from high frequency to low frequency. Since signal components of different frequencies carry different information and functions in the speech signal, the signal characteristics of different frequency bands can be clearly distinguished by sorting, which is convenient for subsequent targeted analysis and processing.

[0052] Using the commonly used frequency analysis methods in the field of signal processing, such as Fourier transform, the frequency analysis of each signal component is performed to determine its main frequency components. Then, according to the frequency size, the signal components are arranged from high to low to form an orderly sequence of signal components.

[0053] (5) Calculate the energies of the plurality of signal components and normalize the energies into relative energies.

[0054] Calculate the energy of multiple signal components. Signal energy is an important indicator to measure signal strength and characteristics. By calculating the energy, we can understand the relative importance of each signal component in the entire speech signal.

[0055] According to the calculation formula of signal energy, the energy of each sorted signal component is calculated. Then, in order to facilitate the energy comparison between different signal components, the calculated energy is normalized to relative energy. The normalization process is to divide the energy of each signal component by the sum of the energies of all signal components to obtain a relative energy value between 0 and 1, which can more intuitively reflect the energy proportion of each signal component in the overall signal.

[0056] (6) Determining an energy entropy of each of the signal components based on the relative energy.

[0057] Based on the normalized relative energy, the energy entropy of each signal component is determined. Energy entropy is a measure of signal uncertainty or randomness. In speech signal processing, it can reflect the uniformity of energy distribution in signal components.

[0058] According to the definition and calculation formula of information entropy, combined with the relative energy of each signal component, the corresponding energy entropy is calculated. The larger the energy entropy, the more uniform the energy distribution in the signal component and the higher the uncertainty; conversely, the smaller the energy entropy, the more concentrated the energy distribution and the more stable the signal.

[0059] (7) The signal component with the largest energy entropy is marked as the target component that dominates the noise.

[0060] The signal component with the largest energy entropy is marked as the target component of the dominant noise. In speech signals, the dominant noise is usually a part with high uncertainty and randomness, and its energy distribution is relatively uniform, and the signal component with the largest energy entropy just meets this characteristic.

[0061] The energy entropy calculation formula is:

[0062] Where, is the relative energy of the i-th signal component, and M is the total number of signal components.

[0063] By comparing the energy entropy of all signal components, the signal component with the largest energy entropy value is found and marked as the target component. This target component will be the focus of processing in the subsequent speech signal denoising and recognition process, and will be suppressed or removed through targeted algorithms and technologies to improve the quality of speech signals and recognition accuracy.

[0064] (8) Perform wavelet transform on the speech signal based on the preset number of decomposition layers and calculate the length L of the wavelet coefficients of each layer.

[0065] The speech signal is subjected to wavelet transform based on the preset number of decomposition layers. In discrete wavelet transform, each layer of decomposition will obtain approximate coefficients (low-frequency part) and detail coefficients (high-frequency part).

[0066] The method for calculating the length of wavelet coefficients at each layer is as follows: Assume that the length of the original speech signal is M. After decomposition at the first layer, the approximate coefficient length L a1 and detail coefficient length L d1 When decomposing at the i-th layer, the approximate coefficients obtained at the i-1th layer are further decomposed to obtain the approximate coefficient length L at the i-th layer. ai and detail coefficient length L di , which are all about half of the length of the approximate coefficients of the i-1th layer. By recursion, the length of the wavelet coefficients after each layer of decomposition can be calculated.

[0067] (9) Extract the noise variance of the target component , and based on the noise variance Calculate the decomposition threshold, the calculation formula is:

[0068] Where i represents the i-th layer of wavelet transform; (10) Constructing a noise reduction function using the decomposition threshold, wherein the noise reduction function includes:

[0069] in, is the jth coefficient of the i-th layer wavelet decomposition component after denoising, is the jth coefficient of the i-th layer wavelet decomposition component, is the adjustment parameter, n is the parameter that determines the shape of the function, and sgn() is the sign function.

[0070] (11) Using the noise reduction function, the signal component arranged before the target component is subjected to noise reduction processing to obtain a noise-reduced high-frequency component.

[0071] (12) Reconstructing the denoised high-frequency component and the signal component arranged after the target component to obtain a denoised speech signal.

[0072] The denoised high-frequency component and the low-frequency signal component arranged after the target component are reconstructed by using the inverse wavelet transform method to obtain the denoised speech signal.

[0073] This noise reduction method effectively eliminates the noise interference in the computer room.

[0074] S102. Speech feature extraction Convert the speech signal from the time domain to the frequency domain, obtain spectrum information by transforming the spectrum, wherein the transforming process includes framing, windowing, and fast Fourier transform; map the spectrum information to the Mel frequency scale, and then extract the Mel frequency cepstrum coefficients by discrete cosine transform operation; Performing linear prediction sharing on the speech signal, estimating vocal tract model parameters of the speech, and converting the vocal tract model parameters into linear prediction cepstrum coefficients; Extracting a phoneme sequence of a speech signal using a hidden Markov model, determining the pronunciation characteristics of the phoneme by analyzing the acoustic pattern of each phoneme in the phoneme sequence, and counting the occurrence frequency and distribution data of the phoneme in the phoneme sequence; integrating the phoneme sequence, the pronunciation characteristics, the occurrence frequency and the distribution data of each phoneme into a phoneme feature sequence; The intonation information, stress information and rhythm information of the speech signal are extracted and integrated into prosodic features.

[0075] S103. Voice feature cache (1) Generate timestamp The generation of timestamps usually takes the starting point of the speech signal as a reference. When the system starts to receive speech data and extract features, a high-precision clock is started, which records the time from the start of the speech to the extraction of each feature. For example, with millisecond-level time accuracy, if the speech begins to be processed at the 100th millisecond after the system is started, and the first speech feature is extracted at the 120th millisecond, then the timestamp of the feature is 120 milliseconds.

[0076] For a continuous sequence of speech features, each feature will be assigned an increasing timestamp in the order in which the features are extracted to ensure the consistency of the time sequence.

[0077] (2) Cache to the specified area The designated cache area is specially set up to store voice features with timestamps, usually a specific space in the system memory. This area has efficient data read and write capabilities and can quickly store and retrieve voice feature data.

[0078] When caching speech features in a specified area, a specific data structure is used to organize the data, such as a hash table or an array. Taking a hash table as an example, the key can be a timestamp and the value is the corresponding speech feature vector. This makes it easy to quickly find and access the corresponding speech features based on the timestamp.

[0079] (3) Check timestamp consistency and handle errors When inputting the speech features and accent information into the second model for further processing, it is necessary to ensure that their timestamps are consistent, otherwise it will cause the input data of the model to not match, affecting the accuracy of the processing results.

[0080] The system will compare the timestamps of the speech features and accent information input into the second model one by one. If the timestamp of the speech features represents the features of a certain time period, but the timestamp of the accent information does not match it, for example, the time range of the speech features is 100-120 milliseconds, while the time range of the accent information is 150-170 milliseconds, then the timestamps are determined to be inconsistent.

[0081] The validation process is automatically performed before data is entered into the model to ensure data accuracy and consistency.

[0082] When a timestamp inconsistency is detected, the system will generate a detailed error message. The error message will include the specific timestamps of the inconsistent voice features and accent information, as well as the cause of the inconsistency. For example, the error message will show "The voice feature timestamp is 100-120 milliseconds, and the accent information timestamp is 150-170 milliseconds. There is a time synchronization problem in the data processing process."

[0083] The generated error information will be promptly fed back to the system administrator or related processing modules for investigation and correction to avoid the impact of erroneous data on subsequent processing.

[0084] In an embodiment of the present invention, based on step S2, an example is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0085] The first model includes: Feature input layer: As the entrance of the first model, the feature input layer plays a vital preprocessing role in the entire model architecture. It is mainly responsible for normalizing and reducing the dimensionality of the input speech features to improve the efficiency and effect of data processing in subsequent network layers.

[0086] Input dimension setting: The input dimension of this layer is explicitly set to 128 dimensions. The choice of this dimension is usually determined based on a large number of experiments and actual application requirements. In the field of speech processing, 128-dimensional speech features can more comprehensively describe the various characteristics of speech signals, such as pitch, timbre, spectrum and other information. These features are obtained through traditional feature extraction methods such as Mel-frequency cepstral coefficients (MFCC), linear prediction cepstral coefficients (LPCC), or advanced features extracted by deep neural networks.

[0087] Normalization: Normalization is one of the important steps in the feature input layer. Since the feature values ​​of different speech samples have different ranges and distributions, this will cause the model to converge more slowly or fall into a local optimal solution during training. Therefore, it is necessary to normalize the input 128-dimensional speech features so that their feature values ​​are distributed in a relatively uniform range. Common normalization methods include minimum-maximum normalization and z-score normalization.

[0088] Dimensionality reduction: In order to reduce the complexity and computational complexity of the data while retaining the main information of the speech features, the feature input layer will reduce the 128-dimensional speech features to 64 dimensions. The dimensionality reduction method uses principal component analysis (PCA).

[0089] Attention mechanism layer: The attention mechanism layer is used in the model to dynamically capture the important parts of speech features, thereby improving the model's attention and processing capabilities for key information.

[0090] Hidden layer dimension: This layer contains a 64-dimensional hidden layer. This dimension matches the output dimension of the feature input layer after dimensionality reduction, ensuring that data can be smoothly transferred between layers. The neurons in the hidden layer perform nonlinear transformations on the input speech features by learning weights, thereby extracting higher-level feature representations.

[0091] Scaled Dot-Product Attention Mechanism: Scaled Dot-Product Attention is an efficient and widely used attention calculation method. Its core idea is to determine the importance weight of each input element by calculating the similarity between the query, key, and value.

[0092] Long Short-Term Memory Network: Long short-term memory (LSTM) is a special type of recurrent neural network that can effectively handle long-term dependencies in sequence data. In the first model, LSTM is used to model the sequence of speech features processed by the attention mechanism to extract richer contextual information.

[0093] Network structure: The number of hidden units in this LSTM network is 256, and the number of layers is 2. Increasing the number of hidden units can improve the expressiveness of the model, enabling it to learn more complex patterns; while increasing the number of network layers can deepen the depth of the model and further mine the hierarchical information in the data.

[0094] Activation function: The forget gate, input gate, and output gate in LSTM use the Sigmoid function as the activation function. The value range of the Sigmoid function is between [0, 1], which can map the input to this interval, thereby effectively controlling the flow of information. Specifically, the forget gate determines how much information of the cell state at the previous moment needs to be forgotten, the input gate determines how much information of the input at the current moment needs to be added to the cell state, and the output gate determines how much information of the cell state at the current moment needs to be output to the hidden state.

[0095] Multi-task learning mechanism: This LSTM network adopts a multi-task learning mechanism, that is, learning multiple related tasks simultaneously in one model. The output layer includes a first branch and a second branch. The first branch is used to output accent information, and the second branch is used to output semantic information. The benefit of multi-task learning is that it can utilize the correlation between different tasks, promote learning from each other, and improve the generalization ability and performance of the model. For example, there is a certain correlation between accent information and semantic information in speech. By learning these two tasks simultaneously, the model can better understand the overall characteristics of speech.

[0096] import torch import torch.nn as nn class LSTMNetwork(nn.Module): def__init__(self, input_dim=64, hidden_dim=256, num_layers=2): super(LSTMNetwork, self).__init__() self.lstm = nn.LSTM(input_dim, hidden_dim,num_layers, batch_first=True) # The first branch is used to output accent information self.accent_output = nn.Linear(hidden_dim, 10) # Assume there are 10 accent categories # The second branch is used to output semantic information self.semantic_output = nn.Linear(hidden_dim, 20) # Assume there are 20 semantic categories defforward(self, x): lstm_output, _ = self.lstm(x) # Get the output of the last time step last_output = lstm_output[-1] accent_info = self.accent_output(last_output)semantic_info = self.semantic_output(last_output) return accent_info,semantic_info.

[0097] The training method of the first model includes: 1. Data Preparation Data collection: Collect a large amount of speech data from computer room managers, including speech features, accent labels, and semantic labels.

[0098] Data partitioning: Divide the collected data into training set, validation set, and test set. The common partition ratios are 70%, 15%, and 15%.

[0099] Data preprocessing: Normalize speech features to ensure that all features have the same scale, which helps the convergence of the model.

[0100] 2. Define the loss function Since the first model adopts a multi-task learning mechanism, it is necessary to define loss functions for accent information and semantic information separately, and then combine them into a total loss function.

[0101] Assuming that there are C categories of accent information, for the i-th sample, its true accent label is yi, and the accent probability distribution predicted by the model is , then the cross entropy loss of accent information is:

[0102] Where N is the number of samples.

[0103] Assume that the semantic information consists of M categories and the true semantic label of the i-th sample is z i , the semantic probability distribution predicted by the model is , then the cross entropy loss of semantic information is:

[0104] The total loss function is:

[0105] in, and is the weight coefficient.

[0106] 3. Choose an optimizer The Adam optimizer is selected, which combines the advantages of AdaGrad and RMSProp and has the characteristics of adaptive learning rate. It can automatically adjust the learning rate during training and accelerate the convergence of the model. The learning rate of the optimizer is initially set to 0.001.

[0107] 4. Training Process import torch import torch.nn as nn import torch.optim as optim # Assume that the FirstModel class has been defined model = FirstModel() criterion_accent =nn.CrossEntropyLoss() criterion_semantic = nn.CrossEntropyLoss() optimizer =optim.Adam(model.parameters(), lr=0.001) alpha = 0.5 beta = 0.5 num_epochs =10 for epoch in range(num_epochs): running_loss = 0.0 for i, (features,accent_labels, semantic_labels) in enumerate(train_loader): optimizer.zero_grad() accent_output, semantic_output = model(features) loss_accent =criterion_accent(accent_output, accent_labels) loss_semantic = criterion_semantic(semantic_output, semantic_labels) loss = alpha * loss_accent + beta* loss_semantic loss.backward() optimizer.step() running_loss += loss.item() print(f'Epoch {epoch + 1}, Loss: {running_loss / len(train_loader)}').

[0108] 5. Model Evaluation Evaluate the performance of the model on the validation set and test set, and use indicators such as accuracy, recall, and F1 value to evaluate the classification effect of accent information and semantic information. Adjust the model's hyperparameters, such as learning rate and weight coefficient, based on the evaluation results.

[0109] The first model is based on a deep learning architecture, combining a feature input layer, an attention mechanism layer, and a long short-term memory network (LSTM). It aims to mine the accent and semantic information contained in the input speech features through multi-level and multi-dimensional analysis and processing. Accent information reflects the speaker's regional or language habits, while semantic information reflects the actual meaning expressed by the speech. The model builds a mapping relationship from speech features to accent and semantic information by learning a large number of speech samples and their corresponding accent and semantic labels.

[0110] In an embodiment of the present invention, based on step S3, an example is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0111] S301. Confirm that the semantic information contains inspection-related instructions.

[0112] In the speech processing process, the first model outputs semantic information through in-depth analysis of speech features. This semantic information is essentially a structured expression of the meaning of the speech content. In order to confirm whether it contains inspection-related instructions, a complete semantic parsing framework needs to be built first. This framework uses natural language processing (NLP) technology, covering multiple links such as lexical analysis, syntactic analysis, and semantic understanding.

[0113] The lexical analysis stage will split the semantic information into independent vocabulary units, and mark these words with parts of speech to clarify their basic attributes in semantic expression. For example, for the semantic information conveyed by voice, "Please arrange personnel to inspect equipment A", the lexical analysis will identify key words such as "arrange", "personnel", "equipment A", and "inspection", and mark "arrange" as a verb, and "personnel" and "equipment A" as nouns.

[0114] Syntactic analysis further explores the grammatical relationship and structure between these words. Through syntactic analysis, we can clarify the subject, predicate, and object components of the sentence, so as to better understand the logical structure of the semantics. In the above example, syntactic analysis will determine that "please arrange" is the predicate part, "personnel" is the object, and "inspect equipment A" is the complement, clearly outlining the semantic framework of the sentence.

[0115] Establishing a dictionary containing inspection-related words and phrases is a key step. This dictionary will be continuously updated and improved to cover various inspection scenarios and expressions. The words in the dictionary may include "inspection", "check", "patrol", "equipment inspection", "area inspection", etc. When parsing the semantic information, the words in it will be matched with the dictionary one by one. If the semantic information contains words or phrases that match those in the dictionary, then it is preliminarily determined that the semantic information contains inspection-related instructions.

[0116] In addition to vocabulary matching, semantic logic judgment is also required to ensure that these matching words do constitute inspection-related instructions. This requires a comprehensive analysis of contextual information and semantic rules. For example, for the semantic information "inspection is an important job", although it contains the word "inspection", it is not a specific inspection instruction, but a statement about inspection work. Therefore, it is necessary to judge whether the sentence expresses the intention to require the execution of inspection actions based on semantic rules. If the sentence contains clear inspection objects (such as equipment, areas, etc.), inspection actions (such as inspection, testing, etc.) and time requirements (such as immediately, tomorrow, etc.), it can be determined that the semantic information contains inspection-related instructions.

[0117] S302. Confirm that inspection-related instructions have authentication requirements.

[0118] Different inspection scenarios correspond to different permission requirements. For example, in an industrial production environment, inspections of key equipment involve the safe operation of the equipment and the confidentiality of production data, so specific permissions are required to execute. For inspections of some ordinary areas, the permission requirements are relatively low. When confirming that inspection-related instructions have authentication requirements, it is necessary to conduct a detailed analysis of the inspection scenario to understand the importance, risk level, and sensitive information involved in the inspection work in this scenario.

[0119] In order to accurately determine whether inspection-related instructions have authentication requirements, it is necessary to establish an authentication rule base. This rule base will define the authentication conditions corresponding to various inspection instructions according to different inspection scenarios and business needs. For example, for inspection instructions involving high-voltage electrical equipment, the authentication rules require that the executor must have an electrical engineer qualification certificate; for inspection instructions involving the enterprise's core data center, the authentication rules require that the executor must undergo special security training and hold valid access credentials.

[0120] When it is confirmed that the semantic information contains inspection-related instructions, the instructions will be matched with the authentication rule base. The system will extract key information from the instructions, such as inspection objects, inspection actions, time requirements, etc., and search for corresponding rules in the authentication rule base based on this information. If a matching rule is found and the rule explicitly requires authentication, then it can be determined that the inspection-related instructions have authentication requirements.

[0121] The authentication rule base is not static, it needs to be dynamically adjusted and updated according to the business changes, security requirements and legal and regulatory updates of the enterprise. For example, as the enterprise introduces new equipment or develops new businesses, the inspection scenarios and permission requirements will change. At this time, the authentication rule base needs to be updated in time to ensure the accuracy and effectiveness of the authentication judgment.

[0122] Through the above steps, it is possible to accurately confirm whether the semantic information contains inspection-related instructions and whether these instructions have authentication requirements. Once it is confirmed that there is an authentication requirement, the system will trigger the corresponding authentication process to ensure the security and compliance of the inspection work.

[0123] In practical applications, this in-depth analysis of semantic information and authentication judgment can effectively ensure the smooth progress of inspection work and avoid security risks and business problems caused by improper permissions. It combines natural language processing technology and permission management strategies to provide a more intelligent and refined solution for enterprise inspection management.

[0124] In an embodiment of the present invention, based on step S4, an example is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0125] The second model includes: Feature fusion layer: The feature fusion layer in the second model has the important task of integrating features from different sources or types. In actual speech processing scenarios, the input features contain multiple dimensions and meanings, and these features can only reflect some of the characteristics of the speech individually. By weighting and summing these input features through the feature fusion layer, their information can be effectively integrated to obtain a more representative and comprehensive fusion feature. This fusion feature can better reflect the overall information contained in the speech and provide a richer and more accurate basis for subsequent identity recognition.

[0126] Weighted summation mechanism: The feature fusion layer performs weighted summation on the input features. In essence, it assigns a specific weight to each input feature, and then multiplies these features by the corresponding weights and adds them together. The size of these weights reflects the importance of each feature in the fusion process. The weights are usually determined through a large number of experiments and training optimization. For example, in a voice identity recognition system, the pitch features, timbre features, and speech speed features of the voice are input. After training, it is found that the pitch feature is more critical in distinguishing different identities. Therefore, when performing weighted summation, the weight corresponding to the pitch feature will be relatively large.

[0127] Support Vector Machines: Support vector machine (SVM) is the core classifier in the second model, responsible for judging the corresponding identity information based on the fusion features output by the feature fusion layer. The basic idea of ​​SVM is to find an optimal classification hyperplane in the feature space so that samples of different categories can be separated as accurately as possible. In voice identity recognition, voice data of different identities can be regarded as samples of different categories. SVM learns the features of these samples and constructs a classification hyperplane, thereby accurately identifying unknown voice identities.

[0128] Kernel function, kernel coefficient and penalty factor: The kernel function is an important component of the support vector machine. Its main function is to map the input low-dimensional features to a high-dimensional space. After mapping it to a high-dimensional space through the kernel function, a linearly separable hyperplane is found for classification. In this model, the polynomial kernel function is used. The polynomial kernel function can flexibly control the properties of the mapped high-dimensional space by adjusting the parameters, thereby adapting to different classification tasks.

[0129] The kernel coefficient plays an important regulatory role in the polynomial kernel function. It controls the distribution of features in high-dimensional space and similarity calculation. A larger kernel coefficient value will reduce the influence range of each sample on the classification hyperplane, making the model more complex and prone to overfitting; while a smaller kernel coefficient value will increase the influence range of the sample, making the model relatively smoother, but will lead to underfitting. Therefore, it is necessary to select the appropriate kernel coefficient through cross-validation method to achieve the best classification effect.

[0130] The penalty factor is another important parameter in the support vector machine. It is used to balance the accuracy of classification and the complexity of the model. In actual classification problems, there will be some misclassified samples. The penalty factor determines the degree of penalty for these misclassified samples. A larger penalty factor value will make the model pay more attention to the accuracy of classification and minimize the number of misclassified samples, but it will make the model too complex and the fit to the training data too high; a smaller penalty factor value will allow more misclassified samples to exist, making the model relatively simpler and more generalizable. Similarly, the penalty factor also needs to be determined through experiments and tuning.

[0131] from sklearn import svm import numpy as np # Generate some sample data X_train = np.random.randn(100, 10) # 100 samples, each with 10-dimensional features y_train = np.random.randint(0, 2, 100) # Binary classification labels # Create a support vector machine classifier # Polynomial kernel function, set the kernel coefficient gamma, penalty factor C and polynomial degree clf = svm.SVC(kernel='poly', gamma=0.1, C=1.0, degree=3) # Training model clf.fit(X_train, y_train) # Prediction example X_test = np.random.randn(10, 10) predictions = clf.predict(X_test)print(predictions).

[0132] The workflow of the second model can be summarized as follows: first, the feature fusion layer receives the input features and fuses them into a more representative fusion feature by weighted summation; then, the support vector machine uses this fusion feature as input, maps it to a high-dimensional space using a polynomial kernel function, and searches for the optimal classification hyperplane in the high-dimensional space; finally, based on this classification hyperplane, the identity information corresponding to the fusion feature is output. Through this workflow, the second model can effectively use the input features for identity recognition and provide accurate identity judgment results for the speech processing system.

[0133] The training process of the second model includes: 1. Data Preparation Data collection: Collect feature data related to the output of the first model and the corresponding identity labels.

[0134] Data division: The data is also divided into training set, validation set and test set.

[0135] Data preprocessing: Perform necessary preprocessing on input features, such as normalization and standardization.

[0136] 2. Feature fusion layer training The weights of the feature fusion layer can be trained by minimizing a certain objective function, such as minimizing the mean square error (MSE) between the fused features and the true target features.

[0137] 3. Support Vector Machine Training Parameter selection: Choose the appropriate kernel coefficient, penalty factor, and polynomial degree. You can use grid search or random search to tune the parameters on the validation set.

[0138] Model training: Use the training set to train the support vector machine.

[0139] from sklearn import svm from sklearn.model_selection importGridSearchCV # Assume that the fused features X_train and identity labels y_train have been obtained parameters ={'kernel': ['poly'], 'gamma': [0.1, 1, 10], 'C': [0.1, 1, 10], 'degree': [2,3, 4]} svc = svm.SVC() clf = GridSearchCV(svc, parameters) clf.fit(X_train,y_train) best_model = clf.best_estimator_ 4. Model Evaluation Evaluate the performance of the second model on the test set, and use indicators such as accuracy and confusion matrix to evaluate the effect of identity recognition. Further adjust the parameters of the model based on the evaluation results.

[0140] The processing flow of the second model: When the cached speech features and the accent information output by the first model are input into the second model, they first enter the feature fusion layer. The main task of this layer is to perform weighted summation of the two different types of features to obtain a more representative and comprehensive fusion feature. During the fusion process, each feature will be assigned a corresponding weight, and the size of these weights reflects the importance of the feature in identity recognition. The determination of weights is usually optimized through a large number of experiments and training, such as using methods such as cross-validation to find the optimal weight combination.

[0141] In this way, the feature fusion layer is able to combine the advantages of speech features and accent information to provide richer and more accurate information for subsequent identity recognition.

[0142] The fused features obtained after processing by the feature fusion layer will be input into the support vector machine (SVM). SVM is a powerful classification algorithm. Its goal is to find an optimal classification hyperplane in the high-dimensional feature space to separate the speech data of different identities. In this model, SVM uses a polynomial kernel function, which maps the input low-dimensional features to the high-dimensional space, making the data that is difficult to separate linearly in the low-dimensional space become separable in the high-dimensional space.

[0143] In the training phase, the SVM learns the parameters of the classification hyperplane based on a large amount of fused feature data with identity labels. In the prediction phase, when a new fused feature is input into the SVM, it determines the identity information corresponding to the feature based on the learned classification hyperplane.

[0144] After the support vector machine classification process, the second model will output the identity information of the voice signal. This identity information can be a specific identity identifier, such as a specific personnel number, or an identity category, such as "employee" or "visitor". The output identity information will provide an important basis for subsequent applications, such as being used in access control systems to determine whether to allow people to enter, and in voice assistant systems to provide personalized services.

[0145] In the whole process, in order to ensure the accuracy and reliability of the results, some error handling and optimization work is required. For example, before inputting the voice features and accent information into the second model, it is necessary to verify whether their timestamps are consistent. If they are inconsistent, an error message is generated to avoid using mismatched data for identity recognition. In addition, the second model can be continuously optimized by continuously collecting new data and adjusting the parameters of the model to improve the accuracy and stability of identity recognition.

[0146] In summary, the cached speech features and the accent information output by the first model are input into the second model, and after a series of processing steps such as feature fusion and support vector machine classification, the identity information to which the speech signal belongs is finally obtained. When confirming the identity of the speech signal, the present invention is based on the speech features of the speech signal and the accent information output by the first model, which expands the judgment basis, thereby further improving the judgment accuracy of the support vector machine.

[0147] In an embodiment of the present invention, based on step S5, an example is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0148] In the inspection workflow driven by voice interaction, after confirming that the semantic information contains inspection-related instructions with authentication requirements and obtaining the identity information corresponding to the voice signal, the next key step is to obtain the permissions corresponding to the identity information, confirm whether this permission meets the authentication requirements, and then decide whether to execute the inspection-related instructions. The following is a detailed expansion of the process: S501. Obtain the authority corresponding to the identity information.

[0149] In order to accurately obtain the permissions corresponding to the identity information, the system needs to be seamlessly connected with the enterprise's internal permission management system. The permission management system is the core platform for the enterprise to centrally manage and control the access rights of employees, equipment or other entities. It usually stores detailed permission data, including information such as the operation permissions, access scope, and time limits of different roles. Through interface calls or data synchronization, the voice interaction system can obtain permission data related to the current identity information from the permission management system.

[0150] After connecting to the permission management system, the first thing to do is to verify the identity information obtained to ensure its authenticity and validity. This involves comparing it with the identity authentication database to check whether the identity information has been legally registered and authorized. After verification, the system will query the permission management system based on the identity information to obtain the permission set corresponding to the identity. This permission set is presented in the form of a list, matrix or tree structure, and describes in detail the operating permissions of the identity in different business scenarios.

[0151] The permission data obtained from the permission management system is complex and multi-dimensional, and needs to be parsed and organized for subsequent judgment and use. For example, permission data contains multiple permission items, each of which has different attributes and parameters. The system needs to clean and structure this data and convert it into a format that is easy to understand and compare. At the same time, the permission data also needs to be classified and summarized, for example, the permissions can be divided into different categories such as device operation permissions, area access permissions, and data viewing permissions, so as to have a clearer understanding of the scope of permissions that the identity has.

[0152] S502. Confirm that the authority meets the authentication requirements.

[0153] After obtaining the permissions corresponding to the identity information, it is necessary to compare them with the authentication requirements of the inspection-related instructions. First, the authentication requirements must be analyzed in detail to clarify the specific permissions required by the inspection instructions. Authentication requirements include specific operating permissions (such as equipment debugging, data modification, etc.), permissions to access specific areas (such as computer rooms, warehouses, etc.), specific qualification certificates (such as electrician certificates, security certificates, etc.), and permissions to perform operations within a specific time range. These requirements exist in the form of text descriptions, regular expressions, or data templates, and the system needs to convert them into quantifiable and comparable standards.

[0154] Match and judge the parsed authentication requirements with the permissions corresponding to the identity information one by one. This involves multiple methods such as exact matching, range matching, and logical combination matching. For example, for the matching of operation permissions, it is necessary to check whether the identity permission set contains the specific operations required by the inspection instruction; for the matching of area access permissions, it is necessary to determine whether the access area allowed by the identity permission covers the area specified in the inspection instruction. In the matching process, it is also necessary to consider restrictions such as the validity period and frequency of use of permissions. If all authentication requirements can find corresponding matches in the identity permission set, it can be considered that the permissions meet the authentication requirements; otherwise, the permissions do not meet the requirements.

[0155] During the permission matching process, various abnormal situations may occur, such as missing permission data, inconsistent permission information, and ambiguous authentication requirements. For these abnormal situations, establish corresponding processing mechanisms. For example, if permission data is missing, try to obtain data from the permission management system again, or prompt the administrator to supplement the data; if permission information is inconsistent, perform data verification and error correction, or feedback the problem to the relevant department for processing; if the authentication requirement is ambiguous, further communicate and confirm with the user who initiated the instruction to clarify the specific permission requirements.

[0156] S503. Execute inspection-related instructions.

[0157] After confirming that the permissions meet the authentication requirements, the system will prepare and schedule the inspection-related instructions. This includes further refining and decomposing the instructions and converting them into specific task steps and operation processes. For example, if the inspection instruction is to inspect the equipment in a certain area, the system will formulate a detailed inspection route and inspection item list based on the distribution and type of the equipment. At the same time, the system will also reasonably arrange the execution personnel and time based on factors such as the urgency of the task and resource availability to ensure that the inspection task can be carried out efficiently and orderly.

[0158] Among them, inspection-related instructions include question-and-answer scenarios and inspection change scenarios.

[0159] For question-and-answer scenarios, see Figure 2, the speech recognition process begins with the speech wake-up stage. Once the system detects the preset wake-up word (Hello Xiaowei), it immediately starts the multi-round dialogue mode and performs speech recognition in real time. During this process, the system monitors the silence duration. If it exceeds the preset maximum silence threshold, it determines that the current dialogue round is over, and then turns off the real-time speech recognition function and waits for the next wake-up signal to be triggered. In each round of dialogue, the system's primary task is to parse the user's intention and distinguish whether it is asking a query question or issuing an action command. For query questions, the system calls the question-and-answer module to generate and feedback the corresponding answer; for action commands, such as navigation requests, the navigation module is activated to execute the command and report the execution result to the user after completion. After completing a round of dialogue processing, the system enters standby mode, ready to receive and process the wake-up word or voice input for the next round of dialogue.

[0160] The second innovation of this application is the design of multi-round conversational interaction function, which enables the robot to understand complex inspection task descriptions and subsequent feedback. Through natural language processing technology, the robot can have multiple rounds of voice interaction with operation and maintenance personnel, not limited to the execution of a single command. For example, operation and maintenance personnel can ask for inspection results, equipment status, alarm information, etc. in real time, and the robot can provide timely feedback and give processing opinions.

[0161] For inspection change scenarios, please refer to Figure 3 In each round of dialogue, the system will first analyze the user's intention. If the user issues an action instruction, such as "Please check cabinet No. 1", the system will immediately send a request to the navigation module to perform the corresponding navigation action (such as Figure 2 ). After sending the request, the system will monitor the return information of the navigation module. When the navigation module receives a request command, its first task is to evaluate whether it is busy with other inspection tasks. If it is busy, it will immediately feedback the "occupy" signal, and then the real-time voice interaction module will inform the user that the current system is "occupied" and suggest that the user try again later. If the navigation module is idle, the large model is started to analyze whether the command contains object keywords. Once the object keyword is confirmed to exist, the system will further check whether the keyword is listed in the list of reachable objects. If the keyword is in the list, the navigation module will plan the route and send a "success" signal to the real-time voice interaction module. The real-time voice interaction module will then inform the user that "the route has been successfully planned and navigation will be started soon" and wait for the "done" signal that the navigation task is completed. On the contrary, if the object keyword does not appear in the list of reachable objects, the navigation module will send a "fail" signal to the voice interaction module, and the real-time voice interaction module will immediately inform the user that "planning failed" and prompt the user to ask again. After navigating to the target point, the inspection task begins and the inspection results are fed back to the real-time voice interaction module. After completing the inspection task, the system returns to standby mode, ready to respond to the wake-up word for the next round of conversation, allowing the user to reactivate the system for interactive Q&A. In addition to abnormal inspection results, the real-time voice interaction module provides users with feedback in different situations with preset audio content played instantly, such as "Cabinet No. 1 has no fault lights abnormalities." When the user's intention is to ask questions, the system will adopt different processing strategies according to the specific situation. If it is judged to be a chat question (for example, "What should I do if cabinet No. 1 is not locked"), the system will directly call the big model for question and answer; for in-library message questions and answers, that is, those questions that have been preset in the knowledge base, the system will first find the most matching answer in the knowledge base, and then input the found answer into the big model for further optimization and answer generation; if the user's intention is to obtain real-time information (such as "What is the current real-time temperature of room No. 3"), the system will first call the third-party interface to obtain the latest real-time data, and then input this data into the big model to generate an accurate and complete answer. After completing a round of questions and answers, the system will start the real-time voice recognition function again, continue to monitor the user's voice input, and perform subsequent processing processes such as maximum silence time judgment to prepare for the next round of dialogue or questions and answers.

[0162] In some embodiments, the voice control system of the computer room inspection robot may include multiple functional modules composed of computer program segments. The computer program of each program segment in the voice control system of the computer room inspection robot may be stored in the memory of a computer device and executed by at least one processor to execute (see Figure 1 Description) The voice control function of the computer room inspection robot.

[0163] In this embodiment, the voice control system of the computer room inspection robot can be divided into multiple functional modules according to the functions it performs, such as Figure 4 As shown. The functional modules of the system may include: a feature extraction module, a first recognition module, a first processing module, a second recognition module, and a second processing module. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0164] A feature extraction module, used for receiving a speech signal and extracting speech features from the speech signal; A first recognition module, used for recognizing the accent information and semantic information of the speech feature by using a first model; A first processing module, used for confirming that the semantic information includes inspection-related instructions, and the inspection-related instructions have authentication requirements; A second recognition module, used for inputting the voice feature and the accent information into a second model to obtain identity information to which the voice signal belongs; A second processing module is used to obtain the authority corresponding to the identity information, confirm that the authority meets the authentication requirements, and execute the inspection-related instructions; The first model includes an attention mechanism and a long short-term memory network; The second model includes a feature fusion layer and a support vector machine.

[0165] Figure 5 The voice control method of the machine room inspection robot provided for the embodiment of the present application can be applied to the device. It can be understood by those skilled in the art that the device structure involved in the embodiment of the present invention does not constitute a limitation on the device, and the device may include more or less components than shown, or combine certain components, or arrange different components. In an embodiment of the present invention, the device includes but is not limited to a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0166] The device 500 may include: a processor 510, a memory 520 and a communication unit 530. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention, and it may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0167] The memory 520 can be used to store the execution inspection related instructions of the processor 510, and the memory 520 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution inspection related instructions in the memory 520 are executed by the processor 510, the device 500 is enabled to perform some or all of the steps in the following method embodiments.

[0168] The processor 510 is the control center of the storage device, and uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs and / or modules stored in the memory 520, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of a plurality of packaged ICs with the same or different functions. For example, the processor 510 can include only a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0169] The communication unit 530 is used to establish a communication channel so that the storage device can communicate with other devices, receive user data sent by other devices or send user data to other devices.

[0170] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0171] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, which is essentially or contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including a number of inspection-related instructions to enable a computer device (which can be a personal computer, a server, or a second device, a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0172] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0173] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0174] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0176] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A voice control method for a machine room inspection robot, characterized in that: include: receiving a speech signal, and extracting speech features from the speech signal; recognizing the accent information and semantic information of the speech feature using the first model; Confirming that the semantic information includes inspection-related instructions, and the inspection-related instructions have authentication requirements; Inputting the speech feature and the accent information into a second model to obtain identity information to which the speech signal belongs; Obtain the authority corresponding to the identity information, confirm that the authority meets the authentication requirements, and execute the inspection-related instructions; The first model includes an attention mechanism and a long short-term memory network; The second model includes a feature fusion layer and a support vector machine.

2. The method according to claim 1, characterized in that A speech signal is received, and speech features are extracted from the speech signal, wherein the speech features include phoneme features and prosodic features, including: Convert the speech signal from the time domain to the frequency domain, obtain spectrum information by transforming the spectrum, wherein the transforming process includes framing, windowing, and fast Fourier transform; map the spectrum information to the Mel frequency scale, and then extract the Mel frequency cepstrum coefficients by discrete cosine transform operation; Performing linear prediction sharing on the speech signal, estimating vocal tract model parameters of the speech, and converting the vocal tract model parameters into linear prediction cepstrum coefficients; Extracting a phoneme sequence of a speech signal using a hidden Markov model, determining the pronunciation characteristics of the phoneme by analyzing the acoustic pattern of each phoneme in the phoneme sequence, and counting the occurrence frequency and distribution data of the phoneme in the phoneme sequence; integrating the phoneme sequence, the pronunciation characteristics, the occurrence frequency and the distribution data of each phoneme into a phoneme feature sequence; The intonation information, stress information and rhythm information of the speech signal are extracted and integrated into prosodic features.

3. The method according to claim 2, characterized in that Receiving a speech signal and extracting speech features from the speech signal includes: Preprocessing the speech signal, wherein the preprocessing includes denoising; Speech features are extracted from the preprocessed speech signal, and the speech features are normalized.

4. The method according to claim 3, characterized in that Preprocessing the speech signal includes: Add random Gaussian white noise of equal length to the time series of speech signal multiple times; After each noise is added, the speech signal after the noise is added is decomposed into a plurality of intrinsic mode function components and a residual component by using the empirical mode decomposition technique to obtain a plurality of groups of decomposed data, each group of the decomposed data including a plurality of intrinsic mode function components and a residual component; The average values ​​of multiple intrinsic mode function components in multiple groups of decomposed data are calculated respectively to obtain multiple signal components; Sorting the plurality of signal components according to a rule from high frequency to low frequency; Calculating the energies of a plurality of the signal components and normalizing the energies into relative energies; determining an energy entropy of each of said signal components based on said relative energies; The signal component with the largest energy entropy is marked as the target component that dominates the noise; Perform wavelet transform on the speech signal based on the preset number of decomposition layers, and calculate the length L of the wavelet coefficients of each layer; Extract the noise variance of the target component , and based on the noise variance Calculate the decomposition threshold, the calculation formula is: Where i represents the i-th layer of wavelet transform; The decomposition threshold is used to construct a noise reduction function, wherein the noise reduction function comprises: in, is the jth coefficient of the i-th layer wavelet decomposition component after denoising, is the jth coefficient of the i-th layer wavelet decomposition component, is the adjustment parameter, n is the parameter that determines the shape of the function, and sgn() is the sign function; Using the noise reduction function to perform noise reduction processing on the signal component arranged before the target component to obtain a noise-reduced high-frequency component; The noise reduction high frequency component and the signal component arranged after the target component are reconstructed to obtain a noise reduction speech signal.

5. The method according to claim 1, characterized in that: The method further comprises: Generating a timestamp for the voice feature, and caching the voice feature with the timestamp in a designated area; Marking the timestamps of the speech features to which the accent information and the semantic information belong; Check whether the timestamps of the speech features and accent information input into the second model are consistent, and if they are inconsistent, generate an error message.

6. The method according to claim 1, characterized in that The first model includes: A feature input layer, wherein the input dimension of the feature input layer is 128 dimensions, and is used to perform normalization and dimensionality reduction processing on the input speech features, thereby reducing the 128-dimensional speech features to 64 dimensions; Attention mechanism layer, including 64-dimensional hidden layer, using scaled dot product attention; The long short-term memory network has 256 hidden units and 2 layers. The forget gate, input gate and output gate use the Sigmoid function as the activation function. The long short-term memory network adopts a multi-task learning mechanism, and the output layer includes a first branch and a second branch, the first branch is used to output accent information, and the second branch is used to output semantic information.

7. The method according to claim 1, characterized in that The second model includes: The feature fusion layer is used to perform weighted summation of input features to obtain fused features; A support vector machine, used to output identity information corresponding to the fusion feature, the support vector machine including a kernel function, a kernel coefficient and a penalty factor; The kernel function is used to map the input features to a high-dimensional space to find an optimal classification hyperplane in the high-dimensional space, and the kernel function is a polynomial kernel function.

8. A voice control system for a machine room inspection robot, characterized in that: include: A feature extraction module, used for receiving a speech signal and extracting speech features from the speech signal; A first recognition module, used for recognizing the accent information and semantic information of the speech feature by using a first model; A first processing module, used for confirming that the semantic information includes inspection-related instructions, and the inspection-related instructions have authentication requirements; A second recognition module, used for inputting the voice feature and the accent information into a second model to obtain identity information to which the voice signal belongs; A second processing module is used to obtain the authority corresponding to the identity information, confirm that the authority meets the authentication requirements, and execute the inspection-related instructions; The first model includes an attention mechanism and a long short-term memory network; The second model includes a feature fusion layer and a support vector machine.

9. A device, characterized in that: include: A memory, used to store the voice control program of the computer room inspection robot; A processor is used to implement the steps of the voice control method of the computer room inspection robot as described in any one of claims 1 to 7 when executing the voice control program of the computer room inspection robot.

10. A computer-readable storage medium storing a computer program, characterized in that: The readable storage medium stores a voice control program of the machine room inspection robot, and when the voice control program of the machine room inspection robot is executed by the processor, the steps of the voice control method of the machine room inspection robot as described in any one of claims 1 to 7 are implemented.

Images