A voice command recognition method, device and medium for a welcoming and explaining robot

Abstract

The present application discloses a voice command recognition method, device and medium for a welcoming and explaining robot, which is used to solve the technical problem that the existing welcoming and explaining robot cannot perform voice command recognition when offline. The method includes: collecting voice data, and training a preset model based on the voice data to obtain a voice command recognition model; deploying the voice command recognition model on the welcoming and explaining robot; performing VAD detection on the audio data input to the welcoming and explaining robot to obtain a voice command; inputting the voice command into the voice command recognition model to obtain a recognition result corresponding to the audio data.

Description

Technical Field

[0001] The present application relates to the technical field of robot command recognition, and in particular to a method, device and medium for voice command recognition of a welcoming and explaining robot. Background Art

[0002] The welcoming robot is mainly used in museums, cultural centers, exhibition halls and other scenes, mainly for voice explanation. However, if the explanation time is too long or the network signal is poor, it is easy to cause the welcoming robot to go offline, and the robot can no longer continue to explain.

[0003] In order to ensure that the welcoming and explaining robot can also perform normal explaining functions when it is offline, it is considered to control the robot to continue explaining through voice commands. However, the general welcoming and explaining robot is unable to recognize voice commands when it is offline. Summary of the invention

[0004] The embodiments of the present application provide a voice command recognition method, device and medium for a welcoming and explaining robot, so as to solve the technical problem that the existing welcoming and explaining robot cannot perform voice command recognition when it is offline.

[0005] In a first aspect, an embodiment of the present application provides a voice command recognition method for a welcoming and explaining robot, the method comprising: collecting voice data, and training a preset model based on the voice data to obtain a voice command recognition model; deploying the voice command recognition model to the welcoming and explaining robot; performing VAD detection on audio data input into the welcoming and explaining robot to obtain a voice command; inputting the voice command into the voice command recognition model to obtain a recognition result corresponding to the audio data.

[0006] In an example in the present application specification, before training the preset model based on speech data, the method also includes: extracting the Transfomer layer in the first three layers of the DistilBert model to construct a Transfomer model; performing model distillation training on the Transfomer model through the Hubert model; after the training is completed, saving the Transfomer model.

[0007] In an example in the present application specification, a preset model is trained based on voice data, specifically including: preprocessing the voice data to obtain single-channel little-endian audio data of a preset sampling rate; performing VAD detection on the little-endian audio data to obtain training data in sentences; inputting the training data into the preset model for training; after the training is completed, obtaining the voice command recognition model.

[0008] In an example of the present application specification, the voice data is preprocessed, specifically including: performing noise reduction processing and echo elimination processing on the voice data.

[0009] In an example in the present application specification, the training data is input into the preset model for training, specifically including: performing feature extraction on the training data through the FBank algorithm to obtain a feature vector corresponding to the training data; wherein the feature vector is 512 frames and 39 dimensions; inputting the feature vector into a pre-saved Transfomer model for processing; performing feature extraction on the features output by the Transfomer model through 5 Conv1d convolutional layers of different dimensions to obtain global features; performing a max-pooling operation on the global features to obtain a 5-dimensional feature map; and inputting the 5-dimensional feature map into a 64-dimensional FC layer, a 64-dimensional Dropout layer, and a 10-category Softmax layer in sequence to complete the training of the preset model.

[0010] In an example of the present application specification, after completing the training of the preset model, the method further includes: converting the trained preset model into an ONNX model; and quantizing the ONNX model using ONNXruntime to obtain the voice command recognition model.

[0011] In an example of the present application specification, the voice command includes at least any one or more of the following: previous, next, increase volume, decrease volume, stop talking, global explanation, pause explanation, start explanation, continue explanation, and body temperature detection.

[0012] In an example of the present application specification, the collecting of voice data specifically includes: collecting voices of people in preset age groups respectively by waking up six microphones.

[0013] In a second aspect, an embodiment of the present application further provides a voice command recognition device for a welcoming and explaining robot, the device comprising: a processor; and a memory having executable instructions stored thereon, wherein when the executable instructions are executed, the processor executes a voice command recognition method for a welcoming and explaining robot as described above.

[0014] In a third aspect, an embodiment of the present application further provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to execute a voice command recognition method for a welcoming and explaining robot as described above.

[0015] A voice command recognition method and device for a welcoming and explaining robot provided in an embodiment of the present application have the following beneficial effects: by deploying a trained voice command recognition model to the welcoming and explaining robot, it provides the possibility for the robot's voice command recognition, and then voice control of the welcoming and explaining robot can be realized to perform corresponding actions and complete corresponding tasks, thereby realizing that the welcoming and explaining robot can be voice controlled to complete a basic welcoming and guiding function of the robot even when it is offline. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A flow chart of a voice command recognition method for a welcoming and explaining robot provided in an embodiment of the present application;

[0018] Figure 2 A structural diagram of a voice command recognition device for a welcoming and explaining robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0020] The embodiments of the present application provide a voice command recognition method, device and medium for a welcoming and explaining robot, which mainly solves the problem of how to enable the robot to have a simple command word recognition function if the robot is disconnected from the Internet, so that the robot can be voice-controlled to complete a robot's basic welcoming and explaining functions, as well as the robot's commonly used commands, such as volume control, stop talking, etc.

[0021] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0022] Figure 1 A flow chart of a voice command recognition method for a welcoming and explaining robot provided in an embodiment of the present application. Figure 1 As shown, the voice command recognition method in the embodiment of the present application mainly includes the following execution steps:

[0023] Step 101: Obtain a voice command recognition model through voice data training.

[0024] The welcoming and explaining robot in the embodiment of the present application realizes the function of voice command recognition, which is mainly achieved through a voice command recognition model, and the training of the model is extremely important.

[0025] In the embodiment of the present application, the training data of the voice command recognition model is mainly some pre-collected voice data, which is not collected randomly, but is collected specifically for people of different age groups and genders. The specific training process is described below, and the embodiment of the present application is not repeated here.

[0026] Step 102: deploy the voice command recognition model to the welcoming and explaining robot.

[0027] The trained voice command recognition model is deployed on the robot. The deployment here can be achieved through the existing deployment solution, or the voice command recognition model can be written into the robot's control system when the robot leaves the factory / is upgraded, so that the robot can realize voice command recognition.

[0028] Step 103: collect voice commands input into the welcoming and explaining robot.

[0029] A robot deployed with a voice command recognition model can recognize voice commands, but before recognition, voice commands need to be collected first. The voice commands here generally refer to the language spoken by tourists to the welcoming robot. After processing, these languages ​​become voice commands that can be recognized by the robot.

[0030] Step 104: Recognize the voice command using a voice command recognition model.

[0031] After collecting the voice command, the robot can input the voice command into the deployed voice command recognition model to recognize the voice command, and then perform corresponding operations based on the recognition result, so that the robot can complete specific functions even when offline.

[0032] In one implementation of the embodiment of the present application, the collection of voice data can be performed by waking up six microphones to collect audio, and by collecting data from more than 100 people of different ages and genders, a voice data set is obtained. Then, the audio in the aforementioned voice data set is subjected to noise reduction and echo cancellation to generate a single-channel, 16KHZ, 16-bit, little-endian clean audio.

[0033] Furthermore, the clean audio is tested by VAD to determine whether it is a sentence. Since we want to realize the recognition of short sentence commands of the explanation robot, generally a command is a sentence, so VAD is needed to judge the sentence, and then this sentence can be used as a voice command and put into the voice command recognition model for recognition.

[0034] In addition, according to market demand, the commonly used voice commands in the embodiments of the present application include: previous, next, increase volume, decrease volume, stop talking, global explanation, pause explanation, start explanation, continue explanation and body temperature detection, etc., 10 command word recognition problems.

[0035] In a possible implementation of the embodiment of the present application, the training of the voice command recognition model can be implemented through the following process:

[0036] 1) Improve the DistilBert model, retain only the first three layers of the Transformer layer of the DistilBert model, and build a Transformer model. Perform model distillation training on the improved Transformer model through the Hubert model. After the training is completed, save the trained 3-layer Transformer network.

[0037] 2) Perform VAD detection on the clean audio with single channel, sampling rate of 16KHZ, 16bit and little endian that has been collected to generate audio sentence by sentence.

[0038] 3) Perform FBank feature extraction on each sentence of audio to obtain a 512-frame, 39-dimensional vector.

[0039] 4) Input the extracted FBank feature vector into the 3-layer Transformer network pre-trained in the first step.

[0040] 5) Use 5 Conv1d convolutional layers of different dimensions to perform global feature extraction on the features output in step 4 to obtain global features, and then perform max-pooling operation on the aforementioned global features to obtain a 5-dimensional feature map.

[0041] 6) The 5-dimensional feature map is sequentially connected to a 64-dimensional FC layer, a 64-dimensional Dropout layer, and a 10-category Softmax layer to complete the training.

[0042] 7) After training, convert the model into an ONNX model, then use ONNXruntime to quantize the ONNX model to obtain the voice command recognition model, and finally deploy it on the robot.

[0043] The above is a method embodiment of the present application. Based on the same inventive concept, the present application also provides a voice command recognition device for a welcoming and explaining robot, and its structure is as follows: Figure 2 shown.

[0044] Figure 2 This is a structural diagram of a voice command recognition device for a welcoming and explaining robot provided in an embodiment of the present application. Figure 2 As shown, the device includes: a processor; and a memory, on which executable instructions are stored. When the executable instructions are executed, the processor executes the voice command recognition method for the welcoming and explaining robot as described above.

[0045] In a possible implementation of an embodiment of the present application, the processor is used to collect voice data, and train a preset model based on the voice data to obtain a voice command recognition model; deploy the voice command recognition model to a welcoming and explaining robot; perform VAD detection on the audio data input into the welcoming and explaining robot to obtain a voice command; input the voice command into the voice command recognition model to obtain a recognition result corresponding to the audio data.

[0046] In addition, an embodiment of the present application also provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to: collect voice data, and train a preset model based on the voice data to obtain a voice command recognition model; deploy the voice command recognition model to a welcoming and explaining robot; perform VAD detection on the audio data input into the welcoming and explaining robot to obtain a voice command; input the voice command into the voice command recognition model to obtain a recognition result corresponding to the audio data.

[0047] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0048] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0049] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A voice command recognition method for a welcoming and explaining robot, characterized in that: The method comprises: Collect voice data, and obtain a voice command recognition model based on the voice data training, including: extracting the Transfomer layer in the first three layers of the DistilBert model, constructing a Transfomer model, performing model distillation training on the Transfomer model through the Hubert model, and saving the Transfomer model after the training is completed; performing noise reduction and echo cancellation on the voice data to obtain single-channel small-endian audio data with a preset sampling rate, performing VAD detection on the small-endian audio data to obtain training data in units of sentences; performing feature extraction on the training data through the FBank algorithm to obtain a feature vector corresponding to the training data, wherein the The feature vector is 512 frames and 39 dimensions. The feature vector is input into the Transformer model for processing. The features output by the Transformer model are extracted through 5 Conv1d convolutional layers of different dimensions to obtain global features. The global features are subjected to max-pooling operation to obtain a 5-dimensional feature map. The 5-dimensional feature map is sequentially input into a 64-dimensional FC layer, a 64-dimensional Dropout layer, and a 10-classification Softmax layer to complete the model training. The trained model is converted into an ONNX model, and the ONNX model is quantized using ONNXruntime to obtain the voice command recognition model. Deploy the voice command recognition model to the welcoming and explaining robot; Perform VAD detection on the audio data input to the welcoming explanation robot to obtain a voice command, wherein the voice command includes at least one or more of the following: previous, next, increase volume, decrease volume, stop talking, global explanation, pause explanation, start explanation, continue explanation, and body temperature detection; The voice command is input into the voice command recognition model to obtain a recognition result corresponding to the audio data.

2. A method for voice command recognition for a welcoming and explaining robot according to claim 1, characterized in that: The collecting of voice data specifically includes: By waking up six microphones, voices of people in preset age groups are collected separately.

3. A voice command recognition device for a welcoming and explaining robot, characterized in that: The device comprises: processor; And, a memory having executable instructions stored thereon, which, when executed, causes the processor to execute a voice command recognition method for a welcoming and explaining robot as described in any one of claims 1-2.

4. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to execute a voice command recognition method for a welcoming and explaining robot as described in any one of claims 1-2.

Images