Voice interrupt processing method and device, electronic equipment, and storage medium

Abstract

This application provides a voice interruption processing method, apparatus, electronic device, and storage medium, belonging to the field of artificial intelligence technology. The method includes: acquiring a voice interruption command for an electronic device; determining the matching degree between the voice interruption command and a predicted command; the predicted command is a command predicted at a first moment, where the first moment is the moment preceding the current moment; if the matching degree between the voice interruption command and the predicted command is greater than a matching degree threshold, executing the voice interruption command based on the pre-operation result corresponding to the predicted command; the pre-operation result is obtained after preprocessing the predicted command. The voice interruption processing method, apparatus, electronic device, and storage medium provided by this application can improve the response speed to user commands.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and more specifically, relates to a voice interruption processing method and device, electronic device, and storage medium. Background Technology

[0002] The application of voice interaction technology in electronic devices has brought convenience to people. In scenarios where users' hands are occupied, such as driving, exercising, and cooking, users can complete core functions without touching the device (such as buttons or screens) simply by using voice commands, greatly improving the user experience. However, while an electronic device is performing a task, users may issue voice commands to interrupt the current task or start a new one. The loading process of new tasks (such as calling applications, querying data, or starting services) can cause a certain time delay, thus affecting the user experience. Summary of the Invention

[0003] The purpose of this application is to provide a voice interruption processing method, device, electronic device, and storage medium to improve the response speed to user commands.

[0004] A first aspect of this application provides a voice interruption processing method, including:

[0005] Obtain a voice interruption command for an electronic device, and determine the matching degree between the voice interruption command and the predicted command; the predicted command is a command predicted at a first moment, where the first moment is the moment before the current moment.

[0006] If the matching degree between the voice interruption command and the predicted command is greater than the matching degree threshold, the voice interruption command is executed based on the pre-operation result corresponding to the predicted command; the pre-operation result is obtained after preprocessing the predicted command.

[0007] The prediction method of the prediction instruction includes:

[0008] Acquire historical instruction sequences and spatial scene data of the electronic device at a first moment; the historical instruction sequence includes control instructions for the electronic device at the first moment and N historical control instructions for the electronic device prior to the first moment;

[0009] Feature extraction is performed on the historical instruction sequence and the spatial scene data respectively to obtain user behavior feature vector and spatial scene feature vector;

[0010] The user behavior feature vector and the spatial scene feature vector are fused to obtain a fused feature vector.

[0011] The fused feature vector is input into the classifier to obtain the prediction instruction.

[0012] A second aspect of this application provides a voice interruption processing device, comprising:

[0013] The instruction acquisition module is used to acquire voice interruption instructions for electronic devices and determine the matching degree between the voice interruption instructions and the predicted instructions; the predicted instructions are instructions predicted at a first moment, where the first moment is the moment before the current moment.

[0014] The instruction execution module is configured to, when the matching degree between the voice interruption instruction and the predicted instruction is greater than a matching degree threshold, obtain the pre-operation result corresponding to the predicted instruction, wherein the pre-operation result is obtained after preprocessing the predicted instruction; and execute the voice interruption instruction based on the pre-operation result.

[0015] Wherein, when determining the prediction instruction, the instruction acquisition module is used to:

[0016] Acquire historical instruction sequences and spatial scene data of the electronic device at the first moment;

[0017] Feature extraction is performed on the historical instruction sequence and the spatial scene data respectively to obtain user behavior feature vector and spatial scene feature vector; the historical instruction sequence includes the control command for the electronic device at a first moment and N historical control commands for the electronic device before the first moment;

[0018] The user behavior feature vector and the spatial scene feature vector are fused to obtain a fused feature vector.

[0019] The fused feature vector is input into the classifier to obtain the prediction instruction.

[0020] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described voice interruption processing method.

[0021] In a fourth aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described voice interruption processing method.

[0022] The beneficial effects of the voice interruption processing method and apparatus, electronic device, and storage medium provided in this application are as follows:

[0023] This application embodiment obtains a predicted instruction based on the historical instruction sequence and spatial scene data prior to the voice interruption instruction, and performs preprocessing operations on the predicted instruction in advance. When the user's voice interruption instruction is received, the matching degree between the voice interruption instruction and the predicted instruction is first judged. When the matching degree is high, it indicates that the prediction result is correct, and the voice interruption instruction is executed directly using the preprocessing result. This avoids the time delay caused by loading new tasks and improves the response speed to user instructions. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic flowchart of a voice interruption processing method provided in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of the iterative training process of a classifier provided in an embodiment of this application;

[0027] Figure 3 This is a structural block diagram of a voice interruption processing device provided in an embodiment of this application;

[0028] Figure 4 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0031] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a voice interruption processing method provided in an embodiment of this application, which can be executed by an electronic device. The method may include:

[0032] S101: Obtain the voice interruption command for the electronic device and determine the matching degree between the voice interruption command and the predicted command; the predicted command is the command predicted at the first moment, and the first moment is the moment before the current moment.

[0033] The prediction methods for prediction instructions include:

[0034] Acquire the historical instruction sequence and the spatial scene data of the electronic device at the first moment; the historical instruction sequence includes the control instruction for the electronic device at the first moment and N historical control instructions for the electronic device before the first moment; N is a natural number, and 3≤N≤20;

[0035] Feature extraction was performed on historical instruction sequences and spatial scene data respectively to obtain user behavior feature vectors and spatial scene feature vectors;

[0036] The user behavior feature vector and the spatial scene feature vector are fused to obtain a fused feature vector;

[0037] The fused feature vector is input into the classifier to obtain the prediction instruction.

[0038] In this embodiment, the moment when the electronic device receives the voice interruption command (i.e., the time when the user issues the voice interruption command) is determined as the current moment. Considering that user behavior has continuity (such as asking about clothing after checking the weather), periodicity (such as listening to music before bed), and correlation (such as checking routes after opening a map), the pattern of user behavior can be obtained by analyzing the order and frequency of commands in the historical command sequence before the current moment before the electronic device receives the voice interruption command. Based on this, the control command at the current moment can be predicted to obtain the predicted command.

[0039] Furthermore, considering that user needs will vary significantly in different spatial scenarios, such as office scenarios often involving document and meeting-related instructions, home scenarios often involving appliance control and entertainment needs, and outdoor scenarios tending towards navigation and weather inquiries, spatial scenario data can provide constraints for instruction prediction, narrowing down the possible range of instructions and making the prediction more in line with the user's current true intentions.

[0040] Specifically, the first moment is the moment before the current moment, and the first moment is a set time interval (e.g., 5 seconds) from the current moment. Spatial scene data can include environmental noise (the noise intensity can be detected by a microphone), location information (obtained through GPS positioning data), and movement speed (obtained through an accelerometer). In this embodiment, features can be extracted from the historical instruction sequence and the spatial scene data of the first moment to obtain user behavior feature vectors and spatial scene feature vectors. Then, the user behavior feature vectors and spatial scene feature vectors are fused to obtain a fused feature vector. The fused feature vector is input into a classifier to predict the probability distribution of the user's instruction category at the current moment, and the instruction category with the highest probability is determined as the predicted instruction.

[0041] In the process of extracting features from historical instruction sequences, each historical instruction can be converted into a word embedding vector. The temporal correlation can be captured by a Long Short-Term Memory (LSTM) network. The last hidden state is taken as the base vector and then concatenated with statistical features such as instruction frequency to obtain the user behavior feature vector. In the process of extracting features from spatial scene data, numerical data can be normalized and location information can be regionally encoded to obtain the spatial scene feature vector.

[0042] When an electronic device receives a voice interruption command from a user, it can calculate the matching degree between the voice interruption command and the predicted command, and determine whether the prediction result is correct based on the matching degree. Specifically, the matching degree can be obtained by calculating the semantic similarity (such as cosine similarity) between the voice interruption command and the predicted command. For example, if the predicted command is "play music" and the actual voice interruption command is "play a song", calculating the semantic similarity between "play music" and "play a song" can determine that they have the same intent, and the prediction result is correct.

[0043] S102: If the matching degree between the voice interruption command and the predicted command is greater than the matching degree threshold, obtain the pre-operation result corresponding to the predicted command. The pre-operation result is obtained after preprocessing the predicted command; execute the voice interruption command based on the pre-operation result.

[0044] In this embodiment, before receiving the semantic interruption instruction, a preprocessing operation for the predicted instruction can be performed in advance. For example, if the predicted instruction is "play music", the corresponding preprocessing operation can be preloading music data.

[0045] Based on this, a matching threshold can be preset. When the matching degree between the voice interruption command and the predicted command is greater than the matching degree threshold, it indicates that the matching degree between the voice interruption command and the predicted command is high, and the prediction result is correct. At this time, the voice interruption command can be executed directly using the pre-operation result, reducing the time delay caused by loading new tasks and improving the response speed to user commands.

[0046] When the matching degree between the voice interruption command and the predicted command is less than or equal to the matching degree threshold, it indicates that the matching degree between the voice interruption command and the predicted command is low and the prediction result is incorrect. At this time, it is necessary to load resources according to the needs of the voice interruption command, and at the same time, quickly release the preloaded resources to avoid occupying memory.

[0047] As can be seen from the above, the embodiments of this application obtain the predicted instruction based on the historical instruction sequence and spatial scene data before the voice interruption instruction, and perform preprocessing operations on the predicted instruction in advance. When the user's voice interruption instruction is received, the matching degree between the voice interruption instruction and the predicted instruction is first judged. When the matching degree between the two is high, it indicates that the prediction result is correct, and the voice interruption instruction is executed directly using the preprocessing result, which avoids the time delay caused by the loading of new tasks and improves the response speed to user instructions.

[0048] In one embodiment of this application, feature fusion is performed on user behavior feature vectors and spatial scene feature vectors to obtain a fused feature vector, including:

[0049] The attention module calculates the first weight corresponding to the user behavior feature vector and the second weight corresponding to the spatial scene feature vector, and then performs a weighted summation of the user behavior feature vector and the spatial scene feature vector based on the first weight and the second weight to obtain the fused feature vector.

[0050] In this embodiment, calculating the first weight of the user behavior feature vector using the attention module can highlight key user behavior information; calculating the second weight of the spatial scene feature vector using the attention module can highlight key scene information. Linearly combining the user behavior feature vector and the spatial scene feature vector according to their respective weights can highlight features with greater predictive value in the current scene.

[0051] As can be seen from the above, this embodiment dynamically assigns weights to user behavior feature vectors and spatial scene feature vectors based on the attention module, which can highlight key features (such as high-frequency behaviors and strongly correlated spatial features in the current scene), avoid interference from irrelevant features, and help improve the overall prediction accuracy.

[0052] In one embodiment of this application, calculating the first weight corresponding to the user behavior feature vector based on the attention module includes:

[0053] The spatial scene feature vector is projected onto the same dimensional space as the user behavior feature vector, and then added to the user behavior feature vector to obtain the corrected user behavior feature vector.

[0054] Attention is calculated on the corrected user behavior feature vector to obtain the first weight corresponding to the user behavior feature vector.

[0055] In this embodiment, considering that certain user behaviors are more likely to occur in specific spatial scenarios—for example, in a conference room scenario, "meeting minutes" related instructions are more likely to appear—this embodiment uses the spatial scenario feature vector as a bias term when calculating the first weight corresponding to the user behavior feature vector based on the attention module, in order to correct the user behavior feature vector. Specifically, the first weight corresponding to the user behavior feature vector can be calculated using the following formula:

[0056] ;

[0057] in, This represents the first weight corresponding to the user behavior feature vector. This represents the hidden state vector at the current time step t (such as the user behavior feature vector output by LSTM). Represents the feature vector of a spatial scene. This is a weight matrix used to weight the spatial scene feature vectors. Projected onto Same dimensional space, This represents the corrected user behavior feature vector. Represents the historical hidden state vector. This indicates the transpose operation. The vector dimension represents the user behavior feature vector.

[0058] As can be seen from the above, this embodiment takes into account the correlation between spatial scene features and user behavior features, corrects the user behavior feature vector based on the spatial scene feature vector, and calculates the first weight corresponding to the user behavior feature vector based on the corrected user behavior feature vector, so that the first weight more accurately reflects the importance of scene adaptation behavior and improves the reliability of subsequent instruction prediction.

[0059] In one embodiment of this application, the method for determining N includes:

[0060] The spatial scene at the first moment is determined based on the spatial scene feature vector at the first moment.

[0061] If the spatial scene at the first moment is different from the spatial scene at the previous moment, initialize N to the minimum value;

[0062] If the spatial scene at the first moment is the same as the spatial scene at the previous moment, and the value of N is less than the maximum limit, then N is added to the length of the first step.

[0063] In this embodiment, if the value of N is set too small, there will be fewer historical instructions, making it difficult to capture long-term correlations and affecting the continuity of instruction prediction; if the value of N is set too large, it will lead to excessive focus on historical instructions and reduce sensitivity to the current scenario. Therefore, this embodiment sets a minimum and a maximum limit to restrict the range of N values.

[0064] Considering that user behavior will change when the spatial scene changes (e.g., switching from an "office" scene to a "running" scene), the length of the historical instruction sequence (i.e., the value of N) can be initialized to a minimum value to reduce interference from irrelevant historical information.

[0065] In each control cycle of the electronic device, when the spatial scene does not change, the value of N can be gradually increased in the first step. When the value of N reaches the maximum limit after multiple accumulations, the accumulation stops.

[0066] As can be seen from the above, this embodiment initializes the length of the historical instruction sequence to a minimum value when the spatial scene changes, so as to reduce the interference of irrelevant behaviors in the old scene, quickly adapt to the new scene, and improve the prediction accuracy.

[0067] In one embodiment of this application, determining the spatial scene at a first moment based on spatial scene feature vectors includes:

[0068] Calculate the similarity between the spatial scene feature vector and the feature vectors of multiple samples in the feature library; each sample feature vector is assigned a corresponding spatial scene identifier.

[0069] The spatial scene at the first moment is determined based on the spatial scene identifier with the highest similarity.

[0070] In this embodiment, multiple sample feature vectors corresponding to various spatial scenes can be pre-set. Based on this, the similarity between the spatial scene feature vector and multiple sample feature vectors in the feature library is calculated using similarity calculation methods such as cosine similarity and Euclidean distance. The scene identifier (such as "running scene") corresponding to the sample feature vector with the highest similarity is used as the determination result of the current scene.

[0071] As can be seen from the above, this embodiment judges spatial scenes based on similarity matching, without the need for a complex model training process. Scene classification can be quickly achieved by pre-storing multiple sample feature vectors, and the feature library can be updated simply by adding new scene samples, which facilitates system iteration and expansion to new scenes.

[0072] Please refer to Figure 2 In one embodiment of this application, the classifier is trained based on multiple training samples, and the training process includes:

[0073] Perform the iterative operation multiple times until the stopping condition is met;

[0074] The iterative operations include:

[0075] Calculate the cross-entropy loss for each training sample;

[0076] The prediction error rate for each spatial scene is calculated. The prediction error rate for each spatial scene is the ratio between the number of incorrect predictions output by the classifier and the total number of predictions for that spatial scene.

[0077] Obtain the error prediction tolerance and the number of training samples for each spatial scene;

[0078] The penalty weight for each spatial scene is determined based on the prediction error rate, the tolerance for incorrect predictions, and the number of training samples.

[0079] The cross-entropy loss of each training sample is weighted and summed based on the penalty weights corresponding to multiple spatial scenes to obtain the loss function value of the classifier; the parameters of the classifier are then adjusted based on the loss function value.

[0080] The stopping conditions are: the loss function value is less than the error threshold, or the number of iterations reaches the set number.

[0081] In this embodiment, a classifier is obtained by training multiple training samples. Each training sample includes a set of historical instruction sequences, spatial scene data, and corresponding speech interruption instructions. During the training process of the classifier, the parameters of the classifier are adjusted through multiple iterations until the loss function value is less than the error threshold or the number of iterations reaches a set number.

[0082] Cross-entropy loss is used to characterize the difference between the predicted probability distribution of each training sample and the true label. The smaller the cross-entropy loss, the more accurate the prediction. Considering the differences in spatial scenes, in order to improve the robustness and practicality of the classifier in complex scenarios, this embodiment determines the penalty weight corresponding to each spatial scene based on the prediction error rate, tolerance for incorrect predictions, and the number of training samples. The cross-entropy loss of each training sample is weighted and summed based on the penalty weights corresponding to multiple spatial scenes to obtain the loss function value of the classifier. The higher the prediction error rate of a spatial scene, the worse the classifier performs in that scene, and the larger the corresponding penalty weight; the lower the tolerance for incorrect predictions in a spatial scene (i.e., the more sensitive it is to errors), the larger the corresponding penalty weight; the smaller the number of samples in a spatial scene, the larger the penalty weight. Determining the loss function value of the classifier based on the penalty weight allows the classifier to prioritize optimizing scenes with high error rates, ensuring the accuracy of highly sensitive scenes, while alleviating the problem of neglecting niche scenes due to imbalanced samples, ultimately achieving balanced and high-quality predictions across all scenes.

[0083] Specifically, the loss function value of the classifier can be calculated using the following formula:

[0084] ;

[0085] in, Represents the loss function. Indicates the true label, This represents the cross-entropy loss of the i-th training sample. Let represent the predicted probability that the i-th sample belongs to the n-th instruction category, and C represent the number of instruction categories. Let represent the penalty weight of the spatial scene in which the i-th sample is located, and M represent the number of samples.

[0086] As can be seen from the above, this embodiment determines the penalty weight corresponding to each spatial scene based on the prediction error rate, the tolerance of the wrong prediction result, and the number of training samples. Based on the penalty weights corresponding to multiple spatial scenes, the cross-entropy loss of each training sample is weighted and summed. This allows the classifier to pay more attention to scenes with high error rates, few samples, or low tolerance for wrong results, thereby reducing the misclassification rate of the classifier in the above scenarios and improving the robustness and practicality of the classifier in complex scenarios.

[0087] In one embodiment of this application, the penalty weight corresponding to the spatial scene is determined based on the prediction error rate, the tolerance for incorrect prediction results, and the number of training samples for each spatial scene, including:

[0088] Determine the first ratio of the prediction error rate to the tolerance of the prediction error for each spatial scene, and based on the first ratio, determine the error severity assessment value for that spatial scene.

[0089] Determine a second ratio between the total number of training samples and the number of training samples corresponding to each spatial scene, and based on the second ratio, determine the sample sparsity evaluation value corresponding to the spatial scene.

[0090] The penalty weight for each spatial scenario is obtained by weighted summing of the error severity assessment value and the sample sparsity assessment value.

[0091] This embodiment provides a specific implementation method for determining the penalty weight corresponding to each spatial scene.

[0092] First, the error severity assessment value for any spatial scene j can be determined using the following formula:

[0093] ;

[0094] in, This represents the error severity assessment value corresponding to the j-th spatial scene. This represents the prediction error rate corresponding to the j-th spatial scene. This represents the tolerance for incorrect predictions for the j-th spatial scene. For example, if the prediction error rate for the j-th spatial scene is 15% and the tolerance is 10%, then the error severity assessment value is 1.5, requiring a stronger penalty.

[0095] Meanwhile, the sample sparsity evaluation value corresponding to this spatial scene can be determined by the following second formula:

[0096] ;

[0097] in, This represents the sample sparsity evaluation value corresponding to the j-th spatial scene. This represents the number of training samples corresponding to the j-th spatial scene. This represents the total number of training samples. For example, if the total number of training samples is 10,000 and the number of training samples for the j-th spatial scene is 200, then the evaluation value is log(50)≈3.91. The fewer the samples, the larger the evaluation value.

[0098] In this embodiment, a logarithmic function is used to calculate the sample sparsity evaluation value, which can avoid the excessive amplification of weights in small sample space scenarios, while maintaining sensitivity to data scarcity. For example, when the number of samples increases from 100 to 200, the evaluation value decreases from log(100)≈4.61 to log(50)≈3.91, thus reasonably reducing the penalty.

[0099] Based on this, the penalty weight corresponding to each spatial scene can be obtained by weighted summing of the error severity assessment value and the sample sparsity assessment value.

[0100] Specifically, the penalty weight of the spatial scene in which the i-th sample is located can be calculated using the following formula:

[0101] ;

[0102] in, , All are weighting coefficients, and .

[0103] As can be seen from the above, this embodiment comprehensively considers three dimensions—classification error probability, tolerance, and sample sparsity—to calculate the penalty weight for each spatial scene, which makes the calculation of the penalty weight more in line with actual needs.

[0104] Corresponding to the voice interruption processing method in the above embodiment, Figure 3 This is a structural block diagram of a voice interruption processing device provided according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 3 The voice interruption processing device 20 includes an instruction acquisition module 21 and an instruction execution module 22.

[0105] The instruction acquisition module 21 is used to acquire the voice interruption instruction for the electronic device and determine the matching degree between the voice interruption instruction and the predicted instruction; the predicted instruction is the instruction predicted at the first moment, and the first moment is the moment before the current moment.

[0106] The instruction execution module 22 is used to obtain the pre-operation result corresponding to the predicted instruction when the matching degree between the voice interruption instruction and the predicted instruction is greater than the matching degree threshold. The pre-operation result is obtained after preprocessing the predicted instruction. The voice interruption instruction is executed based on the pre-operation result.

[0107] Specifically, when determining the prediction instruction, the instruction acquisition module is used for:

[0108] Acquire the historical instruction sequence and the spatial scene data of the electronic device at the first moment; the historical instruction sequence includes the control instruction for the electronic device at the first moment and N historical control instructions for the electronic device before the first moment; N is a natural number, and 3≤N≤20;

[0109] Feature extraction was performed on historical instruction sequences and spatial scene data respectively to obtain user behavior feature vectors and spatial scene feature vectors;

[0110] The user behavior feature vector and the spatial scene feature vector are fused to obtain a fused feature vector;

[0111] The fused feature vector is input into the classifier to obtain the prediction instruction.

[0112] In one embodiment of this application, the instruction acquisition module 21 is specifically used for:

[0113] The attention module calculates the first weight corresponding to the user behavior feature vector and the second weight corresponding to the spatial scene feature vector, and then performs a weighted summation of the user behavior feature vector and the spatial scene feature vector based on the first weight and the second weight to obtain the fused feature vector.

[0114] In one embodiment of this application, the instruction acquisition module 21 is further configured to:

[0115] The spatial scene feature vector is projected onto the same dimensional space as the user behavior feature vector, and then added to the user behavior feature vector to obtain the corrected user behavior feature vector.

[0116] Attention is calculated on the corrected user behavior feature vector to obtain the first weight corresponding to the user behavior feature vector.

[0117] In one embodiment of this application, the instruction acquisition module 21 is specifically used for:

[0118] The spatial scene at the first moment is determined based on the spatial scene feature vector at the first moment.

[0119] If the spatial scene at the first moment is different from the spatial scene at the previous moment, initialize N to the minimum value;

[0120] If the spatial scene at the first moment is the same as the spatial scene at the previous moment, and the value of N is less than the maximum limit, then N is added to the length of the first step.

[0121] In one embodiment of this application, the instruction acquisition module 21 is further configured to:

[0122] Calculate the similarity between the spatial scene feature vector and the feature vectors of multiple samples in the feature library; each sample feature vector is assigned a corresponding spatial scene identifier.

[0123] The spatial scene at the first moment is determined based on the spatial scene identifier with the highest similarity.

[0124] In one embodiment of this application, the instruction acquisition module 21 is specifically used for:

[0125] Perform the iterative operation multiple times until the stopping condition is met;

[0126] The iterative operations include:

[0127] Calculate the cross-entropy loss for each training sample;

[0128] The prediction error rate for each spatial scene is calculated. The prediction error rate for each spatial scene is the ratio between the number of incorrect predictions output by the classifier and the total number of predictions for that spatial scene.

[0129] Obtain the error prediction tolerance and the number of training samples for each spatial scene;

[0130] The penalty weight for each spatial scene is determined based on the prediction error rate, the tolerance for incorrect predictions, and the number of training samples.

[0131] The cross-entropy loss of each training sample is weighted and summed based on the penalty weights corresponding to multiple spatial scenes to obtain the loss function value of the classifier; the parameters of the classifier are then adjusted based on the loss function value.

[0132] The stopping conditions are: the loss function value is less than the error threshold, or the number of iterations reaches the set number.

[0133] In one embodiment of this application, the instruction acquisition module 21 is further configured to:

[0134] Determine the first ratio of the prediction error rate to the tolerance of the prediction error for each spatial scene, and determine the error severity assessment value for that spatial scene based on the first ratio;

[0135] Determine a second ratio between the total number of training samples and the number of training samples corresponding to each spatial scene, and determine the sample sparsity evaluation value corresponding to the spatial scene based on the second ratio;

[0136] The penalty weight for each spatial scenario is obtained by weighted summing of the error severity assessment value and the sample sparsity assessment value.

[0137] See Figure 4 , Figure 4 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 4The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments, for example... Figure 3 The functions of the instruction acquisition module 21 and instruction execution module 22 are shown.

[0138] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0139] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0140] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store matching thresholds, minimum limits, maximum limits, and error thresholds.

[0141] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the voice interruption processing method provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0142] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0143] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0144] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0145] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0146] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connections shown or discussed may be indirect coupling or communication connections through some interfaces or units, or they may be electrical, mechanical, or other forms of connection.

[0147] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0148] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0149] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for handling voice interruption, applied to electronic devices, characterized in that, include: Obtain a voice interruption command for an electronic device, and determine the matching degree between the voice interruption command and the predicted command; The predicted instruction is the instruction predicted at the first moment, where the first moment is the moment before the current moment; If the matching degree between the voice interruption command and the predicted command is greater than the matching degree threshold, the voice interruption command is executed based on the pre-operation result corresponding to the predicted command. The pre-operation result is obtained after preprocessing the prediction instruction; The prediction method of the prediction instruction includes: Acquire the historical instruction sequence and the spatial scene data of the electronic device at the first moment; the historical instruction sequence includes the control instruction for the electronic device at the first moment and N historical control instructions for the electronic device before the first moment; N is a natural number, and 3≤N≤20; Feature extraction is performed on the historical instruction sequence and the spatial scene data respectively to obtain user behavior feature vector and spatial scene feature vector; The user behavior feature vector and the spatial scene feature vector are fused to obtain a fused feature vector. The fused feature vector is input into the classifier to obtain the prediction instruction; The step of fusing the user behavior feature vector and the spatial scene feature vector to obtain a fused feature vector includes: The attention module calculates the first weight corresponding to the user behavior feature vector and the second weight corresponding to the spatial scene feature vector, and then performs a weighted summation of the user behavior feature vector and the spatial scene feature vector based on the first weight and the second weight to obtain the fused feature vector.

2. The voice interruption processing method as described in claim 1, characterized in that, The first weight corresponding to the user behavior feature vector is calculated based on the attention module, including: The spatial scene feature vector is projected onto the same dimensional space as the user behavior feature vector, and then added to the user behavior feature vector to obtain the corrected user behavior feature vector. Attention is calculated on the modified user behavior feature vector to obtain the first weight corresponding to the user behavior feature vector.

3. The voice interruption processing method as described in claim 1, characterized in that, The methods for determining N include: The spatial scene at the first moment is determined based on the spatial scene feature vector at the first moment. If the spatial scene at the first moment is different from the spatial scene at the previous moment, initialize N to the minimum value; If the spatial scene at the first moment is the same as the spatial scene at the previous moment, and the value of N is less than the maximum limit, then N is incremented by the first step length.

4. The voice interruption processing method as described in claim 3, characterized in that, Determining the spatial scene at the first moment based on the spatial scene feature vector includes: Calculate the similarity between the spatial scene feature vector and multiple sample feature vectors in the feature library; each sample feature vector is assigned a corresponding spatial scene identifier. The spatial scene at the first moment is determined based on the spatial scene identifier with the highest similarity.

5. The voice interruption processing method as described in claim 1, characterized in that, The classifier is trained based on multiple training samples, and the training process includes: Perform the iterative operation multiple times until the stopping condition is met; The iterative operation includes: Calculate the cross-entropy loss for each training sample; The prediction error rate for each spatial scene is calculated; the prediction error rate for each spatial scene refers to the ratio between the number of incorrect predictions output by the classifier and the total number of predictions for that spatial scene. Obtain the error prediction tolerance and the number of training samples for each spatial scene; The penalty weight for each spatial scene is determined based on the prediction error rate, the tolerance for incorrect prediction results, and the number of training samples. The cross-entropy loss of each training sample is weighted and summed based on the penalty weights corresponding to multiple spatial scenes to obtain the loss function value of the classifier; the parameters of the classifier are adjusted based on the loss function value. The stopping condition is: the loss function value is less than the error threshold, or the number of iterations reaches the set number.

6. The voice interruption processing method as described in claim 5, characterized in that, The step of determining the penalty weight corresponding to each spatial scene based on the prediction error rate, the tolerance of the incorrect prediction result, and the number of training samples includes: Determine a first ratio of the prediction error rate to the error prediction tolerance for each spatial scene, and determine the error severity assessment value for that spatial scene based on the first ratio; Determine a second ratio between the total number of training samples and the number of training samples corresponding to each spatial scene, and determine the sample sparsity evaluation value corresponding to the spatial scene based on the second ratio; The penalty weight corresponding to each spatial scene is obtained by weighted summing of the error severity assessment value and the sample sparsity assessment value.

7. A voice interruption processing device, characterized in that, include: The instruction acquisition module is used to acquire voice interruption instructions for electronic devices and determine the matching degree between the voice interruption instructions and the predicted instructions. The predicted instruction is the instruction predicted at the first moment, where the first moment is the moment before the current moment; The instruction execution module is configured to, when the matching degree between the voice interruption instruction and the predicted instruction is greater than a matching degree threshold, obtain the pre-operation result corresponding to the predicted instruction, wherein the pre-operation result is obtained after preprocessing the predicted instruction; and execute the voice interruption instruction based on the pre-operation result. Wherein, when determining the prediction instruction, the instruction acquisition module is used to: Acquire historical instruction sequences and spatial scene data of the electronic device at the first moment; Feature extraction is performed on the historical instruction sequence and the spatial scene data respectively to obtain user behavior feature vector and spatial scene feature vector; the historical instruction sequence includes the control command for the electronic device at the first moment and N historical control commands for the electronic device before the first moment; N is a natural number, and 3≤N≤20; The user behavior feature vector and the spatial scene feature vector are fused to obtain a fused feature vector. The fused feature vector is input into the classifier to obtain the prediction instruction; The instruction acquisition module is specifically used for: The attention module calculates the first weight corresponding to the user behavior feature vector and the second weight corresponding to the spatial scene feature vector, and then performs a weighted summation of the user behavior feature vector and the spatial scene feature vector based on the first weight and the second weight to obtain the fused feature vector.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images