Voice instruction processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a voice instruction processing method and device, electronic equipment and a storage medium, and the method comprises the steps: converting a target voice instruction into a target text instruction if the voice assistant detects the target voice instruction when the voice assistant of a target vehicle is in a wake-up state; inputting the target text instruction into a first voice recognition processing model deployed in the target vehicle, and outputting a target instruction type corresponding to the target text instruction; calling a target voice recognition processing model corresponding to the target instruction type to determine a vehicle control instruction corresponding to the target text instruction; and controlling the target vehicle according to the vehicle control instruction. Through the embodiment of the invention, the text instruction classification can be realized based on the first voice recognition processing model, and then the corresponding voice recognition processing model is called according to the type of the text instruction to determine the vehicle control instruction, so that the vehicle can be accurately controlled according to the voice instruction.

Description

Technical Field

[0001] The present invention relates to the technical field of voice data processing, and in particular to a voice instruction processing method and device, electronic equipment, and storage medium. Background Art

[0002] Human-vehicle interaction is the core of user experience. The fragmented layout of functional areas in traditional car cockpits and information overload have created obstacles to human-vehicle interaction, resulting in the underestimation of the value of the car itself as an interactive entrance. As electronic information technology begins to move into the car, a smart cockpit is created. The smart cockpit can meet the different needs of different people in the car through various intelligent means, which will bring a more intelligent and safe interactive experience. It is also a key interface for new generation technologies such as advanced driver assistance, autonomous driving and artificial intelligence. After the user enters the smart cockpit, the environmental changes in the smart cockpit will affect the accuracy of the user's voice commands. In the car-mounted end, slot recognition and other methods are used to perform intent recognition. The user must have a clear intention to extract the corresponding keywords. If the user issues a more generalized command, the vehicle cannot recognize it. For example, if the user says "I'm so hot", slot recognition cannot determine the function the user wants to control. How to provide users with better command interaction is an urgent problem to be solved. Summary of the invention

[0003] In view of the above problems, a method and device for processing voice commands, an electronic device, and a storage medium are proposed to overcome the above problems or at least partially solve the above problems, including:

[0004] A method for processing a voice command, the method comprising:

[0005] When the voice assistant of the target vehicle is in an awake state, if the voice assistant detects a target voice command, converting the target voice command into a target text command;

[0006] Inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle, and outputting a target instruction type corresponding to the target text instruction;

[0007] Calling a target speech recognition processing model corresponding to the target instruction type to determine a vehicle control instruction corresponding to the target text instruction, wherein the target speech recognition processing model outputs a vehicle control instruction corresponding to the target text instruction based on an input target text instruction;

[0008] The target vehicle is controlled according to the vehicle control instruction.

[0009] Optionally, the step of inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle and outputting a target instruction type corresponding to the target text instruction includes:

[0010] Inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle;

[0011] In the first speech recognition processing model, the target text instruction is segmented to generate a plurality of segmentation data;

[0012] A target instruction type corresponding to the target text instruction is output based on the multiple word segmentation data.

[0013] Optionally, outputting a target instruction type corresponding to the target text instruction based on the plurality of word segmentation data includes:

[0014] Vectorize each word segmentation data to obtain first vector data corresponding to each word segmentation data;

[0015] Determine an average value of the plurality of first vector data as a target sentence vector of the target text data;

[0016] Output a target instruction type corresponding to the target text instruction according to the target sentence vector.

[0017] Optionally, the target speech recognition processing module includes a first speech recognition processing model deployed in the target vehicle and a second speech recognition processing model deployed in the cloud, and calling the target speech recognition processing model corresponding to the target instruction type to determine the vehicle control instruction corresponding to the target text instruction includes:

[0018] When the target instruction type is a terminal-side instruction type, calling a first speech recognition processing model deployed in the target vehicle to determine a vehicle control instruction corresponding to the target text instruction;

[0019] When the instruction type is a cloud-side instruction type, the target text instruction is sent to the cloud to which the target vehicle is connected, so as to determine the vehicle control instruction corresponding to the target text instruction using a second speech recognition processing model deployed on the cloud.

[0020] Optionally, calling a first speech recognition processing model deployed in the target vehicle to determine a vehicle control instruction corresponding to the target text instruction includes:

[0021] Determining a target word slot text corresponding to the target text instruction based on a first speech recognition processing model deployed in the target vehicle;

[0022] A vehicle control instruction corresponding to the target text instruction is generated according to the target word slot text.

[0023] Optionally, the training process of the first speech recognition processing model includes:

[0024] Obtaining sample text instructions corresponding to sample voice instructions used for training the first speech recognition processing model;

[0025] Determining an expected instruction type corresponding to the sample text instruction;

[0026] Inputting the sample text instruction into a preset initial speech recognition model to perform instruction classification prediction, and obtaining a predicted instruction type corresponding to the sample text instruction;

[0027] Determine a first loss function value of a preset first loss function based on the expected instruction type and the predicted instruction type;

[0028] Adjusting model parameters of the initial speech recognition model according to the first loss function value;

[0029] When the adjusted speech recognition model meets the preset training termination condition, outputting the adjusted speech recognition model;

[0030] When the adjusted speech recognition model does not meet the preset training termination condition, iterative training is performed based on the adjusted speech recognition model.

[0031] Optionally, it also includes:

[0032] When the expected instruction type is a terminal-side instruction type, determining an expected word slot text corresponding to the sample text instruction;

[0033] Inputting the sample text instruction into a preset initial speech recognition model to perform word slot text prediction, and obtaining a predicted word slot text corresponding to the sample text instruction;

[0034] Determine a second loss function value of a preset second loss function based on the expected word slot text and the predicted word slot text;

[0035] The step of adjusting the model parameters of the initial speech recognition model according to the first loss function value includes:

[0036] Obtain first weight data of the first loss function value and second weight data of the second loss function value, wherein the sum of the first weight data and the second weight data is 1;

[0037] Using the first weight data and the second weight data to perform weighted summation on the first loss function value and the second loss function value, respectively, to obtain a target loss function value;

[0038] The model parameters of the initial speech recognition model are adjusted based on the target loss function value.

[0039] A voice command processing device, the device comprising:

[0040] A target text instruction generating module, configured to convert the target voice instruction into a target text instruction if the voice assistant detects the target voice instruction when the voice assistant of the target vehicle is in an awake state;

[0041] An instruction classification module, used for inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle, and outputting a target instruction type corresponding to the target text instruction;

[0042] A vehicle control instruction generation module, used for calling a target speech recognition processing model corresponding to the target instruction type to determine a vehicle control instruction corresponding to the target text instruction, wherein the target speech recognition processing model outputs a vehicle control instruction corresponding to the target text instruction based on an input target text instruction;

[0043] A vehicle control module is used to control the target vehicle according to the vehicle control instruction.

[0044] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the voice instruction processing method as described above when executed by the processor.

[0045] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the voice instruction processing method described above is implemented.

[0046] The embodiments of the present invention have the following advantages:

[0047] When the voice assistant of the target vehicle is in an awake state, if the voice assistant detects a target voice command, the target voice command is converted into a target text command; the target text command is input into a first voice recognition processing model deployed in the target vehicle, and a target command type corresponding to the target text command is output; the voice recognition processing model corresponding to the target command type is called to determine the vehicle control command corresponding to the target text command; the target vehicle is controlled according to the vehicle control command, so that text command classification can be realized based on the first voice recognition processing model, and then the corresponding voice recognition processing model is called according to the type of text command to determine the vehicle control command, so as to realize precise control of the vehicle according to the voice command. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0049] Figure 1a is a flow chart of the steps of a voice command processing method provided by one embodiment of the present invention;

[0050] Figure 1b is a schematic diagram of a training process of a first speech recognition processing model provided by an embodiment of the present invention;

[0051] Figure 2a is a flowchart of another method for processing voice commands provided by an embodiment of the present invention;

[0052] Figure 2b This is a schematic diagram of voice command processing provided by an embodiment of the present invention;

[0053] Figure 3 It is a structural diagram of a voice command processing device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Reference Figure 1a , shows a flow chart of the steps of a voice control method provided by an embodiment of the present invention, which is applied to a smart cockpit of a car and may specifically include the following steps:

[0056] Step S101, when the voice assistant of the target vehicle is in an awake state, if the voice assistant detects a target voice command, the target voice command is converted into a target text command;

[0057] In actual applications, a voice assistant can be deployed in the target vehicle, and the voice assistant is used by the user to control the target vehicle by voice. When the voice assistant of the target vehicle is in the awake state, the voice assistant can monitor the user's voice in the target vehicle, and then when the target voice command is detected, the target voice command can be converted into a target text command through the ASR model (Automatic Speech Recognition) deployed in the target vehicle.

[0058] In one example, an audio acquisition device may be deployed in the target vehicle, and then the audio is collected by the audio acquisition device, and the collected audio is filtered and processed to remove interference, and then the target voice command is extracted from the processed audio data. In the filtering process, interfering ambient sound and invalid user voice data, such as modal particles without any semantic meaning, may be filtered out.

[0059] In the embodiment of the present invention, the target voice command may be a generalized voice command, where the generalized voice command refers to a voice command that is not dependent on a specific context or a specific device.

[0060] Step S102, inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle, and outputting a target instruction type corresponding to the target text instruction;

[0061] A trained first speech recognition processing model can be pre-deployed in the target vehicle. The speech recognition processing model establishes an association between text instructions and instruction types, and then instruction classification can be performed based on the input text instructions to determine the instruction type corresponding to the text instruction.

[0062] Thus, after obtaining the target text instruction, the target text instruction can be input into the first speech recognition processing model, and after being processed by the first speech recognition processing model, the target instruction type corresponding to the target text instruction can be obtained. In an embodiment of the present invention, the target instruction type can be divided into terminal-side instructions and cloud-side instructions, wherein the terminal-side instructions are instructions that can be processed by the target vehicle, and the cloud-side instructions are instructions that can be processed by the cloud. In practical applications, in order to take into account the instruction processing speed, instruction recognition accuracy, and coverage, some relatively simple voice instructions can be divided into terminal-side instructions, which are quickly executed by the target vehicle, and some more complex voice instructions can be converted into cloud-side instructions, which are processed by the cloud. The specific terminal-side instruction or cloud-side instruction division can be set according to the actual scenario requirements.

[0063] In one embodiment of the present invention, inputting a target text instruction into a first speech recognition processing model deployed in a target vehicle and outputting a target instruction type corresponding to the target text instruction include the following sub-steps:

[0064] Step S11, inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle;

[0065] Step S12, in the first speech recognition processing model, performing word segmentation processing on the target text instruction to generate a plurality of word segmentation data;

[0066] In practical applications, word segmentation processing can divide the text into smaller units. Specifically, the word segmentation rules can be set in advance, and then the target text instructions are segmented according to the word segmentation rules to obtain multiple word segmentation data.

[0067] Step S13: outputting a target instruction type corresponding to the target text instruction based on the multiple word segmentation data.

[0068] Specifically, the target instruction type corresponding to the target text instruction is output based on multiple word segmentation data, including: vectorizing each word segmentation data to obtain the corresponding first vector data of each word segmentation data; determining the average value of the multiple first vector data as the target sentence vector of the target text data; and outputting the target instruction type corresponding to the target text instruction according to the target sentence vector.

[0069] Among them, word segmentation can be vectorized in any of the following ways to generate vector data: bag-of-words model, TF-IDF, Word2Vec, FastText, BERT (Bidirectional Encoder Representations from Transformers).

[0070] In one embodiment of the present invention, the training process of the first speech recognition processing model includes:

[0071] Step S21, obtaining a sample text instruction corresponding to a sample voice instruction for training a first speech recognition processing model;

[0072] Step S22, determining the expected instruction type corresponding to the sample text instruction;

[0073] In practical applications, the expected instruction type can be determined by manually annotating each sample text instruction.

[0074] Step S23, inputting the sample text instruction into a preset initial speech recognition model for instruction classification prediction, and obtaining a predicted instruction type corresponding to the sample text instruction;

[0075] The initial speech recognition model can be selected according to actual needs. For example, the basic model can be generated by qwen2-0.5B-instruct.

[0076] Step S24, determining a first loss function value of a preset first loss function based on the expected instruction type and the predicted instruction type;

[0077] Among them, the first loss function can be set according to the actual scenario, and the first loss function is a function with the expected instruction type and the predicted instruction type as variables.

[0078] Step S25, adjusting the model parameters of the initial speech recognition model according to the first loss function value;

[0079] Step S26, when the adjusted speech recognition model meets the preset training termination condition, output the adjusted speech recognition model;

[0080] The training termination condition may be based on the number of training times reaching a preset number, the model accuracy reaching a preset value, etc.

[0081] When the adjusted speech recognition model meets the preset training termination conditions, the model training is stopped and the adjusted speech recognition model is output. The adjusted speech recognition model can be used as the trained first speech recognition processing model, and then the first speech recognition processing model can be deployed on the target vehicle for speech recognition processing.

[0082] Step S27, when the adjusted speech recognition model does not meet the preset training termination condition, iterative training is performed based on the adjusted speech recognition model.

[0083] If the currently adjusted speech recognition model does not meet the preset training termination conditions, it is necessary to continue iterative training based on the adjusted speech recognition model, that is, to replace the initial recognition model with the adjusted speech recognition model to execute steps S23 to S25, so as to realize iterative training of the speech recognition model, until the adjusted speech recognition model after the iterative training meets the preset training termination conditions, then the adjusted speech recognition model after the iterative training is output, and the adjusted speech recognition model after the iterative training can be used as the first speech recognition processing model, and then the first speech recognition processing model can be deployed on the target vehicle for speech recognition processing.

[0084] Step S103, calling a target speech recognition processing model corresponding to the target instruction type to determine a vehicle control instruction corresponding to the target text instruction;

[0085] The target speech recognition processing model outputs a vehicle control instruction corresponding to the target text instruction based on the input target text instruction. In one example, the target speech recognition processing model may include a first speech recognition processing model deployed in the target vehicle and a second speech recognition processing model deployed in the cloud.

[0086] In one embodiment of the present invention, the training process of the first speech recognition processing model also includes: when the expected instruction type is an end-side instruction type, determining the expected word slot text corresponding to the sample text instruction; inputting the sample text instruction into a preset initial speech recognition model for word slot text prediction to obtain the predicted word slot text corresponding to the sample text instruction; and determining the second loss function value of a preset second loss function based on the expected word slot text and the predicted word slot text.

[0087] In practical applications, the first speech recognition model can not only classify text instructions, but also predict the word slot text corresponding to the text instructions.

[0088] When training the first speech recognition model, the corresponding expected word slot text can also be annotated according to the sample text instruction. The word slot text is used to express the vehicle control intention that the user's voice instruction actually needs to express. For example, the user's voice instruction is "It's so hot today." This voice instruction does not have a clear vehicle control instruction, but it actually expresses that the current temperature is too high and the air conditioner needs to be turned on to cool down. Therefore, the word slot text corresponding to the voice instruction is: {'object':'air conditioner', 'action':'turn on'}.

[0089] Among them, the model parameters of the initial speech recognition model are adjusted according to the first loss function value, including: obtaining first weight data of the first loss function value and second weight data of the second loss function value, the sum of the first weight data and the second weight data is 1; using the first weight data and the second weight data to perform weighted summation on the first loss function value and the second loss function value respectively to obtain the target loss function value; based on the target loss function value, the model parameters of the initial speech recognition model are adjusted.

[0090] That is, the target loss function = first weight data * first loss function value + second weight data * second loss function value.

[0091] In practical applications, the training process of the second speech recognition processing model can refer to the first speech recognition processing model.

[0092] Reference Figure 1b , shows a schematic diagram of the training process of a first speech recognition processing model in an embodiment of the present invention. The model structure in the embodiment of the present invention is: the basic model uses the qwen2-0.5B-instruct generation model, and adds a classifier layer for classification on the basis of the qwen2 generation model, so that the model has both generation and classification capabilities.

[0093] The training process of the first speech recognition processing model may include:

[0094] (I) Select training data: Collect simple vehicle control direct commands and simple product-defined scenarios as end-side samples, and determine the word slots corresponding to the samples through annotation; use other complex scenario samples as cloud-side samples, and use the end-side samples and cloud-side samples as classification data for training; cloud-side samples and end-side samples can be expanded, enhanced, and filtered using a large language model.

[0095] (II) Training process: During the training process, taking "It's so hot today" as an example, the instruction text ("It's so hot today") and the expected word slot text ("{'object':'air conditioner', 'action':'open'}") are first concatenated and then segmented by the model's built-in word segmenter. After segmentation, they are converted into vectors and input into the model. Through the feature extraction of the model, the six vectors from [S1] to [E1] are averaged to obtain the sentence vector to represent the semantics of the entire sentence "It's so hot today". The sentence vector is then input into the dense layer (fully connected layer) used for classification for supervised learning of classification, and its loss function is recorded as classification_loss (i.e., the first loss function).

[0096] At the same time, the vectors from [S2] to [T13] are supervised using the slot text as labels, and the loss function is recorded as generate_loss (i.e., the second loss function); thus, the total loss function loss (i.e., the target loss function) is:

[0097] loss=α*classification_loss+(1-α)*genetate_loss

[0098] Among them, 0<α<1. Through the above process, multi-task training and fine-tuning of the above two tasks can be achieved simultaneously.

[0099] Step S104, controlling the target vehicle according to the vehicle control instruction.

[0100] After the vehicle control instruction is obtained, the vehicle control instruction may be sent to a module in the target vehicle for executing the vehicle control instruction so as to execute the vehicle control instruction.

[0101] In an embodiment of the present invention, by integrating a large language model (i.e., a first speech processing model and a target speech processing model) into the vehicle computer and applying it to slot recognition, the user's generalized intention (such as: I am so hot) can be accurately identified as a corresponding instruction, giving the user a more natural interaction rather than an imperative interaction, thereby improving the user experience.

[0102] In an embodiment of the present invention, when the voice assistant of the target vehicle is in an awake state, if the voice assistant detects a target voice command, the target voice command is converted into a target text command; the target text command is input into a first voice recognition processing model deployed in the target vehicle, and a target command type corresponding to the target text command is output; the voice recognition processing model corresponding to the target command type is called to determine a vehicle control command corresponding to the target text command; the target vehicle is controlled according to the vehicle control command, so that text command classification can be implemented based on the first voice recognition processing model, and then the corresponding voice recognition processing model is called according to the type of text command to determine the vehicle control command, so as to realize precise control of the vehicle according to the voice command.

[0103] Reference Figure 2a , shows a flow chart of another voice control method provided by an embodiment of the present invention, which may specifically include the following steps:

[0104] Step S201, when the voice assistant of the target vehicle is in an awake state, if the voice assistant detects a target voice command, the target voice command is converted into a target text command;

[0105] Step S202, inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle, and outputting a target instruction type corresponding to the target text instruction;

[0106] Step S203: When the target instruction type is a terminal-side instruction type, calling a first speech recognition processing model deployed in the target vehicle to determine a vehicle control instruction corresponding to the target text instruction;

[0107] In one embodiment of the present invention, a first speech recognition processing model deployed in a target vehicle is called to determine a vehicle control instruction corresponding to a target text instruction, including: determining a target word slot text corresponding to the target text instruction based on the first speech recognition processing model deployed in the target vehicle; and generating a vehicle control instruction corresponding to the target text instruction based on the target word slot text.

[0108] In actual applications, the first speech recognition processing model can generate a slot text according to the input text instruction. When the instruction type is a terminal-side instruction, the first speech recognition processing model deployed locally can be directly called in the target vehicle to generate a target slot text. Then, the vehicle control instruction that can control the vehicle is obtained according to the slot text.

[0109] In the embodiment of the present invention, the first speech recognition processing model can be obtained by training and improving based on the basic model, so that the large language model with a small number of parameters can be applied to the terminal side for slot recognition, thereby improving the generalization ability of semantic understanding on the vehicle side. At the same time, by modifying the structure of the large language model and integrating the classification function on the basis of the generated model, the storage and computing power consumption required for using additional models for terminal-cloud arbitration is saved, and the time required for word slot prediction is saved.

[0110] Step S204: when the instruction type is a cloud-side instruction type, the target text instruction is sent to the cloud to which the target vehicle is connected, so as to determine the vehicle control instruction corresponding to the target text instruction by using a second speech recognition processing model deployed in the cloud;

[0111] When the instruction type is a cloud-side instruction type, the first speech recognition processing model deployed in the target vehicle cannot obtain the word slot text corresponding to the target text instruction, and the text instruction needs to be sent to the cloud. The second speech recognition processing model in the cloud obtains the vehicle control instruction that can be executed by the target vehicle based on the target word slot text corresponding to the target text instruction, and then based on the target word slot text. The cloud sends the generated vehicle control instruction to the target vehicle to control the target vehicle.

[0112] In an embodiment of the present invention, the end-cloud arbitration task and the intent recognition task can be integrated into one model, avoiding the use of an additional model for end-cloud arbitration, saving the memory and computing power consumption of the additional model; when predicting the word slot after classification, there is no need to re-extract the feature vector corresponding to the input instruction, and only the feature vector extracted during classification needs to be reused, which saves the time for feature extraction and provides users with a faster response, shortening the response time from 2s to 1s.

[0113] Step S205, controlling the target vehicle according to the vehicle control instruction.

[0114] In an embodiment of the present invention, after the collected voice instructions are converted into text instructions, the instruction type is determined by the first speech recognition processing model, and then the corresponding model is called to generate word slot text to obtain vehicle control instructions, thereby improving the accuracy of voice instruction recognition. At the same time, classification processing can alleviate the data processing pressure on the vehicle side.

[0115] Reference Figure 2b, shows a schematic diagram of voice command processing in an embodiment of the present invention. The target vehicle receives the voice command "It's so hot today" issued by the user, converts the voice command into a text command, and then inputs the command text into the end-side model of the target vehicle (i.e., the first voice processing model). The command text is segmented and vectorized, and then the classifier classification layer is used to predict whether the command text is an end-side command. If the command text is an end-side command, the word slot is further predicted in the end-side model to obtain the word slot text, and then the vehicle control command is output. If the command text is not an end-side command, it is input into the cloud-side model in the cloud for recognition to obtain the vehicle control command.

[0116] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0117] Reference Figure 3 , shows a schematic diagram of the structure of a voice command processing device provided by an embodiment of the present invention, which may specifically include the following modules:

[0118] A target text instruction generating module 301 is used for converting the target voice instruction into a target text instruction when the voice assistant of the target vehicle is in an awake state and the voice assistant detects the target voice instruction;

[0119] An instruction classification module 302 is used to input the target text instruction into a first speech recognition processing model deployed in the target vehicle, and output a target instruction type corresponding to the target text instruction;

[0120] A vehicle control instruction generation module 303 is used to call a target speech recognition processing model corresponding to the target instruction type to determine a vehicle control instruction corresponding to the target text instruction, and the target speech recognition processing model outputs a vehicle control instruction corresponding to the target text instruction based on an input target text instruction;

[0121] The vehicle control module 304 is used to control the target vehicle according to the vehicle control instruction.

[0122] In one embodiment of the present invention, the instruction classification module 302 may include the following submodules:

[0123] A target text instruction input submodule, used for inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle;

[0124] A word segmentation processing submodule, used for performing word segmentation processing on the target text instruction in the first speech recognition processing model to generate a plurality of word segmentation data;

[0125] The target instruction type output submodule is used to output the target instruction type corresponding to the target text instruction based on the multiple word segmentation data.

[0126] In one embodiment of the present invention, the target instruction type output submodule may include:

[0127] A word segmentation vectorization processing unit, used to vectorize each word segmentation data to obtain first vector data corresponding to each word segmentation data;

[0128] a sentence vector determining unit, configured to determine an average value of the plurality of first vector data as a target sentence vector of the target text data;

[0129] A target instruction type determination unit is used to output a target instruction type corresponding to the target text instruction according to the target sentence vector.

[0130] In one embodiment of the present invention, the target speech recognition processing module includes a first speech recognition processing model deployed in the target vehicle and a second speech recognition processing model deployed in the cloud, and the vehicle control instruction generation module 303 may include:

[0131] A terminal-side instruction processing submodule, configured to, when the target instruction type is a terminal-side instruction type, call a first speech recognition processing model deployed in the target vehicle to determine a vehicle control instruction corresponding to the target text instruction;

[0132] A cloud-side instruction determination submodule is used to send the target text instruction to the cloud to which the target vehicle is connected when the instruction type is a cloud-side instruction type, so as to determine the vehicle control instruction corresponding to the target text instruction using a second speech recognition processing model deployed on the cloud, and the second speech recognition processing model outputs the vehicle control instruction corresponding to the target text instruction based on the input target text instruction.

[0133] In an embodiment of the present invention, the terminal-side instruction processing submodule may include:

[0134] A target word slot text determination unit, configured to determine a target word slot text corresponding to the target text instruction based on a first speech recognition processing model deployed in the target vehicle;

[0135] The vehicle control instruction generation submodule is used to generate a vehicle control instruction corresponding to the target text instruction according to the target word slot text.

[0136] In one embodiment of the present invention, the device further comprises:

[0137] A sample acquisition module, used to acquire sample text instructions corresponding to sample voice instructions used for training the first speech recognition processing model;

[0138] An expected instruction type determination module, used to determine the expected instruction type corresponding to the sample text instruction;

[0139] An instruction type prediction module is used to input the sample text instruction into a preset initial speech recognition model to perform instruction classification prediction and obtain a predicted instruction type corresponding to the sample text instruction;

[0140] A first loss function value calculation module, used to determine a first loss function value of a preset first loss function based on the expected instruction type and the predicted instruction type;

[0141] A model parameter adjustment module, used for adjusting the model parameters of the initial speech recognition model according to the first loss function value;

[0142] A training termination module, used for outputting the adjusted speech recognition model when the adjusted speech recognition model meets a preset training termination condition;

[0143] The iterative training module is used for performing iterative training based on the adjusted speech recognition model when the adjusted speech recognition model does not meet the preset training termination condition.

[0144] In one embodiment of the present invention, the device further comprises:

[0145] An expected word slot text determination module, used for determining an expected word slot text corresponding to the sample text instruction when the expected instruction type is a terminal-side instruction type;

[0146] A slot text prediction module, used for inputting the sample text instruction into a preset initial speech recognition model to perform slot text prediction, and obtain a predicted slot text corresponding to the sample text instruction;

[0147] A second loss function value calculation module, used to determine a second loss function value of a preset second loss function based on the expected word slot text and the predicted word slot text;

[0148] Wherein, the model parameter adjustment module may include:

[0149] A weight data acquisition submodule, used to acquire first weight data of the first loss function value and second weight data of the second loss function value, wherein the first weight data and the second weight data add up to 1;

[0150] a target loss function value calculation submodule, configured to use the first weight data and the second weight data to perform weighted summation on the first loss function value and the second loss function value, respectively, to obtain a target loss function value;

[0151] The model parameter adjustment submodule is used to adjust the model parameters of the initial speech recognition model based on the target loss function value.

[0152] An embodiment of the present invention further provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the above voice instruction processing method is implemented.

[0153] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above language instruction processing method is implemented.

[0154] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0155] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0156] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0157] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate a device for implementing the functions specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0158] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0160] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0161] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0162] The above is a detailed introduction to a voice command processing method and device, electronic device, and storage medium. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for processing voice commands, characterized in that: The method comprises: When the voice assistant of the target vehicle is in an awake state, if the voice assistant detects a target voice command, converting the target voice command into a target text command; Inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle, and outputting a target instruction type corresponding to the target text instruction; Calling a target speech recognition processing model corresponding to the target instruction type to determine a vehicle control instruction corresponding to the target text instruction, wherein the target speech recognition processing model outputs a vehicle control instruction corresponding to the target text instruction based on an input target text instruction; The target vehicle is controlled according to the vehicle control instruction.

2. The method according to claim 1, characterized in that The step of inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle and outputting a target instruction type corresponding to the target text instruction includes: Inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle; In the first speech recognition processing model, the target text instruction is segmented to generate a plurality of segmentation data; A target instruction type corresponding to the target text instruction is output based on the multiple word segmentation data.

3. The method according to claim 2, characterized in that The outputting the target instruction type corresponding to the target text instruction based on the multiple word segmentation data includes: Vectorize each word segmentation data to obtain first vector data corresponding to each word segmentation data; Determine an average value of the plurality of first vector data as a target sentence vector of the target text data; Output a target instruction type corresponding to the target text instruction according to the target sentence vector.

4. The method according to any one of claims 1 to 3, characterized in that The target speech recognition processing module includes a first speech recognition processing model deployed on the target vehicle and a second speech recognition processing model deployed on the cloud, and calling the target speech recognition processing model corresponding to the target instruction type to determine the vehicle control instruction corresponding to the target text instruction includes: When the target instruction type is a terminal-side instruction type, calling a first speech recognition processing model deployed in the target vehicle to determine a vehicle control instruction corresponding to the target text instruction; When the instruction type is a cloud-side instruction type, the target text instruction is sent to the cloud to which the target vehicle is connected, so as to determine the vehicle control instruction corresponding to the target text instruction using a second speech recognition processing model deployed on the cloud.

5. The method according to claim 4, characterized in that The calling of the first speech recognition processing model deployed in the target vehicle to determine the vehicle control instruction corresponding to the target text instruction includes: Determining a target word slot text corresponding to the target text instruction based on a first speech recognition processing model deployed in the target vehicle; A vehicle control instruction corresponding to the target text instruction is generated according to the target word slot text.

6. The method according to any one of claims 1 to 3, characterized in that The training process of the first speech recognition processing model includes: Obtaining sample text instructions corresponding to sample voice instructions used for training the first speech recognition processing model; Determining an expected instruction type corresponding to the sample text instruction; Inputting the sample text instruction into a preset initial speech recognition model to perform instruction classification prediction, and obtaining a predicted instruction type corresponding to the sample text instruction; Determine a first loss function value of a preset first loss function based on the expected instruction type and the predicted instruction type; Adjusting model parameters of the initial speech recognition model according to the first loss function value; When the adjusted speech recognition model meets the preset training termination condition, outputting the adjusted speech recognition model; When the adjusted speech recognition model does not meet the preset training termination condition, iterative training is performed based on the adjusted speech recognition model.

7. The method according to claim 6, characterized in that Also includes: When the expected instruction type is a terminal-side instruction type, determining an expected word slot text corresponding to the sample text instruction; Inputting the sample text instruction into a preset initial speech recognition model to perform word slot text prediction, and obtaining a predicted word slot text corresponding to the sample text instruction; Determine a second loss function value of a preset second loss function based on the expected word slot text and the predicted word slot text; The step of adjusting the model parameters of the initial speech recognition model according to the first loss function value includes: Obtain first weight data of the first loss function value and second weight data of the second loss function value, wherein the sum of the first weight data and the second weight data is 1; Using the first weight data and the second weight data to perform weighted summation on the first loss function value and the second loss function value, respectively, to obtain a target loss function value; The model parameters of the initial speech recognition model are adjusted based on the target loss function value.

8. A voice command processing device, characterized in that: The device comprises: A target text instruction generating module, configured to convert the target voice instruction into a target text instruction if the voice assistant detects the target voice instruction when the voice assistant of the target vehicle is in an awake state; An instruction classification module, used for inputting the target text instruction into a first speech recognition processing model deployed in the target vehicle, and outputting a target instruction type corresponding to the target text instruction; A vehicle control instruction generation module, used for calling a target speech recognition processing model corresponding to the target instruction type to determine a vehicle control instruction corresponding to the target text instruction, wherein the target speech recognition processing model outputs a vehicle control instruction corresponding to the target text instruction based on an input target text instruction; A vehicle control module is used to control the target vehicle according to the vehicle control instruction.

9. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the voice instruction processing method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the voice instruction processing method according to any one of claims 1 to 7 is implemented.

Images