Voice message processing method and device, computer equipment and storage medium

Abstract

The invention relates to a voice message processing method and device, computer equipment and a storage medium, so as to accurately identify instructions in a plurality of voice messages and avoid the problem that non-instruction voice messages are wrongly matched with vehicle machine instructions or errors are reported due to the fact that the non-instruction voice messages cannot be matched with adaptive vehicle machine instructions. The method comprises the following steps: in response to a received voice message, determining a first probability that the voice message is an instruction based on a text classification model; in response to determining that the person in the vehicle watches the target area and / or the number of the person in the vehicle is 1, determining an amplification factor, and correcting the first probability based on the amplification factor to obtain a second probability; wherein the amplification factor is greater than 1; and in response to the fact that the second probability is greater than a first preset threshold value, determining that the voice message is the instruction.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a method, device, computer equipment and storage medium for processing voice messages. Background Art

[0002] With the development of intelligent driving, the degree of intelligence of vehicles is getting higher and higher, and the voice information that can match the vehicle computer instructions during driving is becoming more and more diverse. In this way, the voice information that can match the vehicle computer instructions is becoming more and more generalized, and is no longer limited to clear instructions.

[0003] Taking turning on the air conditioning in the car as an example, when the user says a highly relevant voice message in the car, such as "the weather is really hot", the car computer command of "turn on the cabin air conditioning" can be matched. At the same time, in order to further improve the user experience and avoid the need for multiple voice wake-ups of the in-car voice assistant between consecutive or short-interval commands, the time window limit for multiple rounds of conversations has gradually been relaxed from the initial 5-8 seconds to about 15 seconds. This leads to the situation that in multi-round conversation scenarios, non-command voice information of passengers and / or drivers in the car is easily mistaken for instructions, resulting in frequent mismatches of non-command voice information with car computer commands, or errors due to the inability to match non-command voice messages with car computer commands. Summary of the invention

[0004] Based on this, it is necessary to provide a voice message processing method, device, computer equipment and storage medium to address the above-mentioned technical problems, so as to accurately identify the instructions in multiple voice messages, avoid incorrectly matching vehicle computer instructions for non-command voice messages, or report errors due to the inability to match non-command voice messages with appropriate vehicle computer instructions.

[0005] In a first aspect, an embodiment of the present application provides a method for processing a voice message, comprising:

[0006] In response to receiving the voice message, determining a first probability that the voice message is an instruction based on a text classification model;

[0007] In response to determining that the person in the vehicle is looking at the target area, and / or the number of the people in the vehicle is 1, determining a magnification factor, and correcting the first probability based on the magnification factor to obtain a second probability; wherein the magnification factor is greater than 1;

[0008] In response to the second probability being greater than a first preset threshold, the voice message is determined to be the instruction.

[0009] In one embodiment, determining that the person in the vehicle is looking at the target area includes:

[0010] Determining the focus area of ​​the line of sight of the person in the vehicle by means of a line of sight capture device;

[0011] In response to an intersection-over-annexation ratio of the focus area to the target area being greater than a second preset threshold, and a duration in which the line of sight of the occupant of the vehicle falls on the focus area being greater than a third preset threshold, it is determined that the occupant of the vehicle is looking at the target area.

[0012] In one embodiment, determining that the person in the vehicle is looking at the target area includes:

[0013] Determine the target person who sent the voice message;

[0014] In response to determining that the target person is looking at the target area, it is determined that the person in the vehicle is looking at the target area.

[0015] In one of the embodiments, the amplification factor is a first amplification factor, a second amplification factor or a third amplification factor;

[0016] Then, in response to determining that the person in the vehicle is looking at the target area and the number of the people in the vehicle is 1, determining the magnification factor includes:

[0017] In response to determining that the person in the vehicle is looking at the target area, and the number of people in the vehicle is 1, determining the first magnification factor;

[0018] or,

[0019] In response to determining that the person in the vehicle is looking at the target area, determining the magnification factor includes:

[0020] In response to determining that the person in the vehicle is looking at the target area, determining the second magnification factor;

[0021] or,

[0022] In response to determining that the number of people in the vehicle is 1, determining an amplification factor includes:

[0023] In response to determining that the number of occupants of the vehicle is 1, the third amplification factor is determined.

[0024] In one embodiment, in response to determining that the person in the vehicle is looking at the target area, determining the second magnification factor includes:

[0025] In response to the person in the vehicle looking at the target area, acquiring a first auxiliary factor;

[0026] The first auxiliary factor is determined as the second amplification factor.

[0027] In one embodiment, after obtaining the first auxiliary factor, the method further includes:

[0028] Determine whether the number of people in the vehicle is 1; if so, obtain a second auxiliary factor, and determine that the product of the first auxiliary factor and the second auxiliary factor is the second amplification factor;

[0029] If not, a third auxiliary factor is obtained, and a product of the first auxiliary factor and the third auxiliary factor is determined as the second amplification factor.

[0030] In a second aspect, an embodiment of the present application provides a message processing device, including:

[0031] a classification module, configured to determine, in response to receiving a voice message, a first probability that the voice message is an instruction based on a text classification model;

[0032] an amplification module, configured to determine an amplification factor in response to determining that a person in the vehicle is looking at a target area and / or the number of people in the vehicle is 1, and to modify the first probability based on the amplification factor to obtain a second probability; wherein the amplification factor is greater than 1;

[0033] The instruction module is used to determine that the voice message is the instruction in response to the second probability being greater than a first preset threshold.

[0034] In one of the embodiments, the magnification module is specifically used to determine the focal area of ​​the line of sight of the person in the vehicle through a line of sight capture device; in response to an intersection ratio of the focal area and the target area being greater than a second preset threshold, and a duration in which the line of sight of the person in the vehicle falls on the focal area being greater than a third preset threshold, it is determined that the person in the vehicle is looking at the target area.

[0035] In one of the embodiments, the amplification module is specifically used to determine a target person who sends the voice message; in response to determining that the target person is looking at the target area, determine that the person in the vehicle is looking at the target area.

[0036] In one embodiment, the amplification factor is a first amplification factor, a second amplification factor or a third amplification factor; the amplification module is further used to determine the first amplification factor in response to determining that the occupant is looking at a target area and the number of occupants is 1; or, the amplification module is further used to determine the second amplification factor in response to determining that the occupant is looking at a target area; or, the amplification module is further used to determine the third amplification factor in response to determining that the number of occupants is 1.

[0037] In one of the embodiments, the amplification module is further used to obtain a first auxiliary factor in response to the person in the vehicle looking at the target area; and determine the first auxiliary factor as the second amplification factor.

[0038] In one embodiment, the amplification module is also used to determine whether the number of people in the vehicle is 1; if so, obtain a second auxiliary factor, and determine that the product of the first auxiliary factor and the second auxiliary factor is the second amplification factor; if not, obtain a third auxiliary factor, and determine that the product of the first auxiliary factor and the third auxiliary factor is the second amplification factor.

[0039] In one embodiment, the device also includes a reduction module, which is specifically used to determine a reduction factor in response to determining that the person in the car is not looking at the target area and the number of people in the car is greater than 1, and to correct the first probability based on the reduction factor to obtain the second probability; wherein the reduction factor is less than 1.

[0040] In one embodiment, the device also includes a training module, which is used to obtain a training sample set and input the training sample set into a model to be trained; wherein the training sample set includes a text sample corresponding to a speech sample and a preset label corresponding to the text sample, and the preset label includes a first label corresponding to the instruction and a second label corresponding to a non-instruction; the parameters in the model to be trained are adjusted using a loss function in the model to be trained until the value of the loss function is less than a fourth preset threshold, thereby obtaining the text classification model; wherein the loss function includes a weight factor greater than 1, and the weight factor is used to increase the weight of the text sample corresponding to the first label.

[0041] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect and any possible implementation manner when executing the computer program.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect of claim 1 and any possible implementation manner are implemented.

[0043] After receiving the voice message, the above-mentioned voice message processing method, device, computer equipment and storage medium generate an amplification factor through the area that the occupants are looking at and the number of occupants in the car to correct the first probability obtained by the text classification model for the binary classification of the voice message, thereby accurately identifying the instructions in the voice message. This method is particularly suitable for after the start of multiple rounds of dialogue but before the end, and can effectively avoid the problem of misidentifying non-command voice messages as commands and mismatching vehicle computer commands, which reduces user experience and consumes power. At the same time, by accurately distinguishing between command and non-command voice messages, unnecessary time can be avoided for matching vehicle computer commands for non-command voice messages; and it can also avoid the problem that when it is impossible to match non-command voice messages to vehicle computer commands that meet the matching conditions, subsequent voice messages are queued for processing, and the processing efficiency of voice messages is significantly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 FIG. 1 is an application environment diagram of a method for processing a voice message in an embodiment;

[0045] Figure 2 A flowchart of a method for processing a voice message in one embodiment;

[0046] Figure 3 A schematic diagram of a flow chart of a method for determining a text processing model in one embodiment;

[0047] Figure 4 A schematic diagram of the structure of a model to be trained in an embodiment;

[0048] Figure 5 A schematic flow chart of the steps of determining a target area that a person in a vehicle is looking at in one embodiment;

[0049] Figure 6 A schematic diagram of a method for distinguishing rejection from instructions based on a voice message processing method in one embodiment;

[0050] Figure 7 is a structural block diagram of a voice message processing device in one embodiment;

[0051] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0052] The technical solution of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0053] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.

[0054] The voice message processing method provided in this application can be applied to Figure 1 In the application environment shown in the figure, the terminal 102 communicates with the server 104 through the network. The terminal 102 can be used to collect voice messages; collect photos in the car to determine the people in the car; capture the sight of the people in the car, and send the collected voice messages, photos in the car, and the captured sight information of the people in the car (including the location information of the sight point) to the server 104 through the network, so that the server 104 can identify whether the voice message is a command by processing the voice message, the photos in the car, and the sight information of the people in the car.

[0055] Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices equipped with or independent of gaze capture devices, and the server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0056] In one embodiment, Figure 2 As shown, a method for providing instructions is provided to distinguish between noise (for example, voice messages collected by a microphone in a vehicle when making a phone call) and instructions, and the method is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0057] Step 201: In response to receiving a voice message, determine a first probability that the voice message is an instruction based on a text classification model.

[0058] Specifically, when the voice collection device / module in the vehicle recognizes that a person in the vehicle is speaking, it starts to receive the voice message and transmits the voice message to the server through the network.

[0059] Once the server starts to receive the voice message, it can start to identify whether the voice message is a command. In response to receiving the voice message, the voice message can be converted into a text message through ASR (Automatic Speech Recognition), and the text message can be input into the text classification model to determine the first probability that the voice message is a command through the text classification model.

[0060] The text classification model is a binary classification model, which can be any voice arbitration model. The text classification model is pre-trained using the vehicle control instruction sets of various vehicle models collected historically, and the scene corpus corresponding to each vehicle control instruction in the vehicle control instruction set as a training data set.

[0061] Step 202 , in response to determining that the person in the vehicle is looking at the target area and / or the number of people in the vehicle is 1, determining a magnification factor, and correcting the first probability based on the magnification factor to obtain a second probability.

[0062] Specifically, the occupants of the vehicle include the occupants of the driver's seat and the occupants of the passenger seat.

[0063] The target area is a physical vehicle control area and / or a virtual vehicle control area.

[0064] The amplification factor is at least greater than 1 and less than 2. Alternatively, the amplification factor may be selected from any value in [1.5, 2).

[0065] This step corrects the first probability by amplifying the factor, and fine-tunes the first probability by using the features contained in multiple modal information to improve the accuracy of the first probability, thereby effectively improving the accuracy of recognizing commands in voice messages.

[0066] The above-mentioned multiple modal information may include information contained in the voice message and line of sight information of people in the car.

[0067] Step 203: In response to the second probability being greater than the first preset threshold, determining that the voice message is an instruction.

[0068] The setting of the first preset threshold can be determined in combination with the vehicle type, text classification model, and in-vehicle multi-round dialogue mechanism, etc. The first preset threshold can be, for example, 0.5, 0.6, 0.7, etc.

[0069] In the above-mentioned voice message processing method, after receiving the voice information, the voice message is processed using the text classification model to achieve binary classification. At the same time, the first probability output by the text classification model is corrected according to the line of sight of the people in the car and the number of people in the car, thereby effectively avoiding the misjudgment of in-car chats and other content as instructions, and improving the accuracy of identifying voice messages as instructions.

[0070] This method is particularly suitable for multi-round conversation scenarios where the voice assistant has been awakened, and because this method has the significant advantage of high accuracy in recognizing commands, the time window for multi-round conversations can be extended to 30-35 seconds based on the voice message processing method, and the accuracy of recognizing commands is not affected.

[0071] The above-mentioned time window for multiple rounds of conversation refers to: in a multiple rounds of conversation scenario, after the voice assistant is awakened, if there is no valid input, the length of time allowed to directly process voice message instructions without having to wake up the voice assistant again.

[0072] In one embodiment, the text classification model is trained in the following way, please refer to Figure 3 :

[0073] Step 301: obtain a training sample set and input the training sample set into the model to be trained.

[0074] The training sample set includes a text sample corresponding to the speech sample and a preset label corresponding to the text sample.

[0075] The text sample can be obtained by processing the speech sample through ASR.

[0076] The preset label includes a first label corresponding to the instruction and a second label corresponding to a non-instruction. Then the text sample may include a valid sample marked with the first label and an invalid sample marked with the second label.

[0077] Among them, the valid samples may include a vehicle control command set of one or more vehicle models, which can be associated with the scene corpus of the commands in the vehicle control command set.

[0078] Invalid samples may include: open source conversation data sets, and human voice samples collected in noisy environments through microphones and other devices that are not related to vehicle control commands.

[0079] Step 302: Use the loss function in the model to be trained to adjust the parameters in the model to be trained until the value of the loss function is less than a fourth preset threshold, thereby obtaining a text classification model.

[0080] In order to solve the problem of label imbalance caused by sample imbalance, the above loss function includes a weight factor greater than 1, and the weight factor is used to enhance the text samples corresponding to the first label, so as to avoid the tilt of the loss function through the weight factor, so that the model to be trained pays more attention to the valid samples corresponding to the first label during the training process, avoids the model to be trained gradually leaning towards invalid samples during the training process, and promotes the model to be trained to improve its recognition ability of valid samples during the training process, thereby further improving the accuracy of the target model output.

[0081] In one embodiment, the structure of the model to be trained is set with reference to RoBERTa (A Robustly Optimized BERTPretraining Approach, an optimized BERT pre-training method). Figure 4 As shown, for the text obtained by converting the voice message (for example, "It's so hot today"), the word segmenter in the model to be trained is used to segment the text.

[0082] Then, "CLS (Classification Marker)" is added to the text header to instruct the model to be trained to classify the text, and "SEP (Segment separator)" is added to the end of the text to separate it from the following text to obtain the text to be processed.

[0083] Next, the text to be processed is input into the encoder through the embedding layer, and feature fusion and calculation are performed to output the feature vector "C" representing the semantics of the text to be processed: T1, T2, T3, T4, T5, Ts,. The encoder is based on selfattention and MLP (Multi-Layer Perceptron) settings.

[0084] Next, the dimension of the feature vector (e.g. 768 dimensions) is reduced to 2 dimensions through a dense (fully connected layer) layer, so that the probability corresponding to the feature value of each dimension can be calculated through a softmax function.

[0085] Because the position in the two-dimensional feature vector corresponds to the category label, after the probability value is calculated, the category label can be used to determine the label corresponding to each probability value according to the position of the feature value: the probability of the first model label corresponding to the instruction, and the probability of the second model label corresponding to the non-instruction.

[0086] The above softmax function can be expressed as follows:

[0087] y is the first model label or the second model label, and x is the feature value.

[0088] For example, the eigenvector is (-2.3, 0.2), the first eigenvalue in the eigenvector corresponds to the first model label 0, and the second eigenvalue corresponds to the second model 1.

[0089] Using the category label, we can see that the probability of the first model label is 0.0759, and the probability of the second model label is 0.9241. Since 0.9241 is greater than 0.0759, the output is the second model label (0.9241).

[0090] Correspondingly, during the training process, the loss function value is calculated using the 0.9241 and the first label in the preset label, as well as the probability values ​​of the model labels obtained from other samples and the corresponding preset labels, and the parameters of the model to be trained are adjusted in reverse.

[0091] For the trained text classification model, the probability in the model label output by the text classification model can be directly used as the aforementioned first probability.

[0092] Furthermore, the number of people in the car is obtained by collecting images inside the car and performing target detection on the images inside the car.

[0093] Further, the following describes the determination of whether the occupant is looking at the target area. Figure 5 .

[0094] Step 501: Determine the focus area of ​​the line of sight of the person in the vehicle through a line of sight capture device.

[0095] Specifically, the gaze capture device includes but is not limited to a Tobii eye tracking device or an eye tracker.

[0096] The sight line capture device determines the focus area of ​​the sight line of the occupant by capturing the sight line of the occupant in the vehicle.

[0097] Step 502: In response to an intersection-over-union ratio of the focus area to the target area being greater than a second preset threshold, and a duration in which the occupant's line of sight falls on the focus area being greater than a third preset threshold, determining that the occupant is looking at the target area.

[0098] Specifically, the intersection-and-joint ratio is calculated by the position parameters of the focus area and the position parameters of the target area. The position parameters of the focus area and the position parameters of the target area are the coordinates of their respective preset key reference points. For example, if the focus area and the target area are both rectangular, then their position parameters are the midpoint coordinates and vertex coordinates of each.

[0099] In one embodiment, in response to an intersection-over-union ratio between the focus area and the target area being less than or equal to a second preset threshold,

[0100] Alternatively, in response to the duration that the line of sight of the occupant of the vehicle falls on the focus area being less than or equal to a third preset threshold, it is determined that the occupant of the vehicle is not looking at the target area.

[0101] Furthermore, the target person who sent the voice message can be determined. This determination method can still be obtained by detecting the collected image / video of the person in the car. For example, it can be achieved by detecting the lip shape changes of the person in the car; for another example, through the sound source localization technology, the position of the person in the car who sent the voice message is located, and the person in the car at this position is determined as the target person.

[0102] Then, in response to the target person looking at the target area, it is determined that the person in the vehicle is looking at the target area. In this way, by distinguishing whether the person in the vehicle looking at the target area is the same person as the person in the vehicle speaking, the accuracy of determining that the voice message is a command is further improved.

[0103] Further, corresponding to the number of people in the vehicle looking at the target area and / or people in the vehicle being 1 for determining the magnification factor in step 202, the magnification factor may be a first magnification factor, a second magnification factor, or a third magnification factor; Examples 1 to 3 are provided below for detailed description:

[0104] Example 1

[0105] In response to determining that the person in the vehicle is looking at the target area and the number of people in the vehicle is 1, determining the magnification factor specifically includes: in response to determining that the person in the vehicle is looking at the target area and the number of people in the vehicle is 1, determining a first magnification factor.

[0106] Furthermore, in a possible implementation manner, the first amplification factor may be determined directly through a preset corresponding relationship.

[0107] In a possible implementation, in response to a person in the vehicle gazing at the target area and the number of people in the vehicle being 1, a first auxiliary factor and a second auxiliary factor are acquired, and a product of the first auxiliary factor and the second auxiliary factor is determined as a second amplification factor.

[0108] Example 2

[0109] In response to determining that the person in the vehicle is looking at the target area, determining the magnification factor, specifically: in response to determining that the person in the vehicle is looking at the target area, determining a second magnification factor.

[0110] Further, in a possible implementation manner, in response to the person in the vehicle looking at the target area, a first auxiliary factor is acquired, and the first auxiliary factor is determined to be the second amplification factor.

[0111] In a possible implementation, after obtaining the first auxiliary factor, the number of people in the vehicle may be further determined, that is, whether the number of people in the vehicle is 1 is determined; if so, the second auxiliary factor is obtained, and the product of the first auxiliary factor and the second auxiliary factor is determined as the second amplification factor.

[0112] If not, a third auxiliary factor is obtained, and a product of the first auxiliary factor and the third auxiliary factor is determined as a second amplification factor.

[0113] Example 3

[0114] In response to determining that the number of people in the vehicle is 1, determining the amplification factor, specifically: in response to determining that the number of people in the vehicle is 1, determining a third amplification factor.

[0115] Further, in a possible implementation manner, in response to determining that the number of people in the vehicle is 1, a second auxiliary factor is acquired, and the second auxiliary factor is determined to be a third amplification factor.

[0116] In a possible implementation, in response to determining that the number of people in the car is 1, it is further determined whether the people in the car are looking at the target area (for example, the vehicle control area). That is, it is determined whether the people in the car are looking at the target area; if so, the first auxiliary factor is obtained, and the product of the first auxiliary factor and the second auxiliary factor is determined to be the third amplification factor.

[0117] If not, a fourth auxiliary factor is obtained, and a product of the fourth auxiliary factor and the second auxiliary factor is determined to be the third amplification factor.

[0118] It can be seen that the first amplification factor, the second amplification factor, and the third amplification factor in the above-mentioned embodiments 1 to 3 can all play a role in fine-tuning the first probability to increase its value, thereby effectively improving the accuracy of the speech recognition result.

[0119] Then the first amplification factor, the second amplification factor and the third amplification factor are all greater than 1 and less than 2. Moreover, the first amplification factor is greater than or equal to the second amplification factor. The first amplification factor is greater than or equal to the third amplification factor.

[0120] Furthermore, the above-mentioned voice message processing method also includes:

[0121] In response to determining that the person in the vehicle is not looking at the target area and the number of people in the vehicle is greater than 1, a reduction factor is determined, and the reduction factor is multiplied by the first probability to obtain a second probability. The reduction factor is less than 1.

[0122] In a possible implementation, the reduction factor is at least greater than 0.5, preferably greater than or equal to 0.6, so as to achieve the purpose of fine-tuning the first probability.

[0123] In a possible implementation, the reduction factor may be determined by a third auxiliary factor and a fourth auxiliary factor. Specifically, in response to determining that the occupant is not looking at the target area and the number of occupants is greater than 1, the product of the third auxiliary factor and the fourth auxiliary factor is determined as the reduction factor.

[0124] Among the first auxiliary factor, the second auxiliary factor, the third auxiliary factor, and the fourth auxiliary factor used to determine the first to third amplification factors and the reduction factor, the fourth auxiliary factor corresponds to the target area that the occupant of the vehicle is not looking at, and the first auxiliary factor corresponds to the target area that the occupant of the vehicle is looking at, so the first auxiliary factor is greater than the fourth auxiliary factor.

[0125] Similarly, the second auxiliary factor corresponds to the number of people in the car being 1, and the third auxiliary factor corresponds to the number of people in the car being greater than 1, so the second auxiliary factor is greater than the third auxiliary factor.

[0126] Furthermore, the first auxiliary factor, the second auxiliary factor, the third auxiliary factor, and the fourth auxiliary factor may be preset values, and the first auxiliary factor is greater than the fourth auxiliary factor, and the second auxiliary factor is greater than the third auxiliary factor.

[0127] Alternatively, in one possible implementation, the fourth auxiliary factor may be determined to be A S , then the first auxiliary factor can be (1+A S ). And, first determine that the third auxiliary factor is A N , then the second auxiliary factor can be (1+A N ).but

[0128] The above reduction factor can be A s *A N .

[0129] And, the first amplification factor in the above-mentioned embodiment 1 is (1+A S )*(1+A N ).

[0130] In the above-mentioned embodiment 2, the second amplification factor may be the first auxiliary factor (1+A S ); or, the second amplification factor may be the product of the first auxiliary factor and the second auxiliary factor: (1+A S )*(1+A N ); or, the second amplification factor may be the product of the first auxiliary factor and the third auxiliary factor: (1+A S )*A N .

[0131] The third amplification factor in the aforementioned embodiment 3 may be the second auxiliary factor (1+A N ); or, the third amplification factor may be the product of the second auxiliary factor and the first auxiliary factor: (1+A S )*(1+A N ); or, the third amplification factor may be the product of the second auxiliary factor and the fourth auxiliary factor: (1+A N )*A S.

[0132] Then the second probability P(C) can be obtained by calculating the function value of the function f(N,S) whose independent variable is the first auxiliary factor, the second auxiliary factor, the third auxiliary factor or the fourth auxiliary factor:

[0133]

[0134] Among them, P 0 (C) is the first probability, S=1 indicates that the person in the car is looking at the target area, N=1 indicates that the number of people in the car is 1, N>1 indicates that the number of people in the car is greater than 1, and S=0 indicates that the person in the car is not looking at the target area.

[0135] In one embodiment, the second probability is determined by the following method: Figure 6 .

[0136] After receiving the voice message, the first probability (P 0 (C)).

[0137] Furthermore, when the eye tracker is used to track the sight of the occupant in the vehicle and fixates on the vehicle control area (i.e., the target area), the first probability (P 0 (C)), and / or by detecting the in-car photos collected by the in-car camera, when it is determined that the number of people in the car is 1, increase the first probability (P 0 (C)), and obtain the second probability P(C).

[0138] When the eye tracker is used to track the sight of the occupants in the car and they are not looking at the car control area (i.e., the target area), the first probability (P 0 (C)), and obtain the second probability P(C). At this time, the probability of rejection (1-P(C)) increases. And / or by detecting the photos of the car collected by the camera inside the car, when it is determined that the number of people in the car is greater than 1, the first probability (P 0 (C)), and obtain the second probability P(C). At this time, the probability of rejection (1-P(C)) is also increased.

[0139] Finally, the second probability P(C) can be compared with the probability of rejection (1-P(C)). Here, rejection refers to a voice message that is not a command.

[0140] For example, when the increased first probability is greater than 0.5, the increased first probability is greater than the probability of rejection, so it can be determined that the voice message is an instruction.

[0141] When the increased first probability is less than or equal to the probability of rejection, the voice message is determined to be rejected.

[0142] Alternatively, when the first probability is greater than the increased probability of rejection, the voice message may be determined to be an instruction. When the first probability is less than or equal to the increased probability of rejection, the voice message is determined to be rejected.

[0143] It should be understood that although Figure 2-Figure 6 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-Figure 6 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0144] Based on the same inventive concept, Figure 7 As shown, the embodiment of the present application provides a voice message processing device, including: a classification module 701, an amplification module 702 and an instruction module 703, wherein:

[0145] The classification module 701 is used to determine, in response to receiving a voice message, a first probability that the voice message is the instruction based on a text classification model.

[0146] The magnification module 702 is used to determine a magnification factor in response to determining that the person in the car is looking at the target area and / or the number of people in the car is 1, and to correct the first probability based on the magnification factor to obtain a second probability; wherein the magnification factor is greater than 1.

[0147] The instruction module 703 is configured to determine that the voice message is the instruction in response to the second probability being greater than a first preset threshold.

[0148] The amplification module 702 is specifically used for:

[0149] The focusing area of ​​the line of sight of the person in the vehicle is determined by a line of sight capture device; in response to an intersection ratio of the focusing area and the target area being greater than a second preset threshold, and a duration in which the line of sight of the person in the vehicle falls on the focusing area being greater than a third preset threshold, it is determined that the person in the vehicle is looking at the target area.

[0150] The amplification module 702 is specifically used for:

[0151] Determine a target person who sends the voice message; and in response to determining that the target person is looking at the target area, determine that the person in the vehicle is looking at the target area.

[0152] The amplification factor is the first amplification factor, the second amplification factor or the third amplification factor; the amplification module 702 is further used for:

[0153] In response to determining that the person in the vehicle is looking at the target area and the number of the people in the vehicle is 1, determining the first magnification factor; or, the magnification module 702 is further used for:

[0154] In response to determining that the person in the vehicle is looking at the target area, determining the second magnification factor; or, the magnification module 702 is further used for:

[0155] In response to determining that the number of occupants of the vehicle is 1, the third amplification factor is determined.

[0156] The amplification module 702 is also used for:

[0157] In response to the person in the vehicle looking at the target area, a first auxiliary factor is acquired; and the first auxiliary factor is determined to be the second amplification factor.

[0158] The amplification module 702 is also used for:

[0159] Determine whether the number of people in the vehicle is 1; if so, obtain a second auxiliary factor, and determine that the product of the first auxiliary factor and the second auxiliary factor is the second amplification factor; if not, obtain a third auxiliary factor, and determine that the product of the first auxiliary factor and the third auxiliary factor is the second amplification factor.

[0160] The device also includes a training module, which is used to:

[0161] Acquire a training sample set, and input the training sample set into a model to be trained; wherein the training sample set includes a text sample corresponding to a speech sample, and a preset label corresponding to the text sample, and the preset label includes a first label corresponding to the instruction, and a second label corresponding to a non-instruction; use a loss function in the model to be trained to adjust the parameters in the model to be trained until the value of the loss function is less than a fourth preset threshold, so as to obtain the text classification model; wherein the loss function includes a weight factor greater than 1, and the weight factor is used to increase the weight of the text sample corresponding to the first label.

[0162] The device further includes a reduction module, wherein the reduction module is specifically configured to:

[0163] In response to determining that the occupant is not looking at the target area and the number of occupants is greater than 1, a reduction factor is determined, and the first probability is corrected based on the reduction factor to obtain the second probability; wherein the reduction factor is less than 1.

[0164] For the specific definition of the voice message processing device, please refer to the definition of the voice message processing method above, which will not be repeated here. Each module in the above-mentioned voice message processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0165] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a text classification model and its training data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for processing a voice message is implemented.

[0166] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0167] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0168] In response to receiving the voice message, determining a first probability that the voice message is the instruction based on a text classification model;

[0169] In response to determining that the person in the vehicle is looking at the target area, and / or the number of the people in the vehicle is 1, determining a magnification factor, and correcting the first probability based on the magnification factor to obtain a second probability; wherein the magnification factor is greater than 1;

[0170] In response to the second probability being greater than a first preset threshold, the voice message is determined to be the instruction.

[0171] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0172] The focusing area of ​​the line of sight of the person in the vehicle is determined by a line of sight capture device; in response to an intersection ratio of the focusing area and the target area being greater than a second preset threshold, and a duration in which the line of sight of the person in the vehicle falls on the focusing area being greater than a third preset threshold, it is determined that the person in the vehicle is looking at the target area.

[0173] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0174] Determine a target person who sends the voice message; and in response to determining that the target person is looking at the target area, determine that the person in the vehicle is looking at the target area.

[0175] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0176] The amplification factor is the first amplification factor, the second amplification factor or the third amplification factor; then

[0177] Then, in response to determining that the person in the vehicle is looking at the target area, and the number of the people in the vehicle is 1, determining the magnification factor includes: in response to determining that the person in the vehicle is looking at the target area, and the number of the people in the vehicle is 1, determining the first magnification factor; or, in response to determining that the person in the vehicle is looking at the target area, determining the magnification factor includes: in response to determining that the person in the vehicle is looking at the target area, determining the second magnification factor; or, in response to determining that the number of the people in the vehicle is 1, determining the magnification factor includes: in response to determining that the number of the people in the vehicle is 1, determining the third magnification factor.

[0178] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0179] In response to the person in the vehicle looking at the target area, a first auxiliary factor is acquired; and the first auxiliary factor is determined to be the second amplification factor.

[0180] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0181] Determine whether the number of people in the vehicle is 1; if so, obtain a second auxiliary factor, and determine that the product of the first auxiliary factor and the second auxiliary factor is the second amplification factor; if not, obtain a third auxiliary factor, and determine that the product of the first auxiliary factor and the third auxiliary factor is the second amplification factor.

[0182] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0183] In response to determining that the occupant is not looking at the target area and the number of occupants is greater than 1, a reduction factor is determined, and the first probability is corrected based on the reduction factor to obtain the second probability; wherein the reduction factor is less than 1.

[0184] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0185] Acquire a training sample set, and input the training sample set into a model to be trained; wherein the training sample set includes a text sample corresponding to a speech sample, and a preset label corresponding to the text sample, and the preset label includes a first label corresponding to the instruction, and a second label corresponding to a non-instruction; use a loss function in the model to be trained to adjust the parameters in the model to be trained until the value of the loss function is less than a fourth preset threshold, so as to obtain the text classification model; wherein the loss function includes a weight factor greater than 1, and the weight factor is used to increase the weight of the text sample corresponding to the first label.

[0186] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0187] In response to receiving the voice message, determining a first probability that the voice message is the instruction based on a text classification model;

[0188] In response to determining that the person in the vehicle is looking at the target area, and / or the number of the people in the vehicle is 1, determining a magnification factor, and correcting the first probability based on the magnification factor to obtain a second probability; wherein the magnification factor is greater than 1;

[0189] In response to the second probability being greater than a first preset threshold, the voice message is determined to be the instruction.

[0190] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0191] The focusing area of ​​the line of sight of the person in the vehicle is determined by a line of sight capture device; in response to an intersection ratio of the focusing area and the target area being greater than a second preset threshold, and a duration in which the line of sight of the person in the vehicle falls on the focusing area being greater than a third preset threshold, it is determined that the person in the vehicle is looking at the target area.

[0192] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0193] Determine a target person who sends the voice message; and in response to determining that the target person is looking at the target area, determine that the person in the vehicle is looking at the target area.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] The magnification factor is the first magnification factor, the second magnification factor or the third magnification factor; then, in response to determining that the person in the vehicle is looking at the target area, and the number of the people in the vehicle is 1, determining the magnification factor includes: in response to determining that the person in the vehicle is looking at the target area, and the number of the people in the vehicle is 1, determining the first magnification factor; or, in response to determining that the person in the vehicle is looking at the target area, determining the magnification factor includes: in response to determining that the person in the vehicle is looking at the target area, determining the second magnification factor; or, in response to determining that the number of the people in the vehicle is 1, determining the magnification factor includes: in response to determining that the number of the people in the vehicle is 1, determining the third magnification factor.

[0196] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0197] In response to the person in the vehicle looking at the target area, a first auxiliary factor is acquired; and the first auxiliary factor is determined to be the second amplification factor.

[0198] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0199] Determine whether the number of people in the vehicle is 1; if so, obtain a second auxiliary factor, and determine that the product of the first auxiliary factor and the second auxiliary factor is the second amplification factor; if not, obtain a third auxiliary factor, and determine that the product of the first auxiliary factor and the third auxiliary factor is the second amplification factor.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0201] In response to determining that the occupant is not looking at the target area and the number of occupants is greater than 1, a reduction factor is determined, and the first probability is corrected based on the reduction factor to obtain the second probability; wherein the reduction factor is less than 1.

[0202] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0203] Acquire a training sample set, and input the training sample set into a model to be trained; wherein the training sample set includes a text sample corresponding to a speech sample, and a preset label corresponding to the text sample, and the preset label includes a first label corresponding to the instruction, and a second label corresponding to a non-instruction; use a loss function in the model to be trained to adjust the parameters in the model to be trained until the value of the loss function is less than a fourth preset threshold, so as to obtain the text classification model; wherein the loss function includes a weight factor greater than 1, and the weight factor is used to increase the weight of the text sample corresponding to the first label.

[0204] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0205] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0206] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for processing a voice message, characterized in that: include: In response to receiving the voice message, determining a first probability that the voice message is an instruction based on a text classification model; In response to determining that the person in the vehicle is looking at the target area, and / or the number of the people in the vehicle is 1, determining a magnification factor, and correcting the first probability based on the magnification factor to obtain a second probability; wherein the magnification factor is greater than 1; In response to the second probability being greater than a first preset threshold, the voice message is determined to be the instruction.

2. The method according to claim 1, characterized in that Determining the target area that the person in the vehicle is looking at includes: Determining the focus area of ​​the line of sight of the person in the vehicle by means of a line of sight capture device; In response to an intersection-over-annexation ratio of the focus area to the target area being greater than a second preset threshold, and a duration in which the line of sight of the occupant of the vehicle falls on the focus area being greater than a third preset threshold, it is determined that the occupant of the vehicle is looking at the target area.

3. The method according to claim 1, characterized in that Determining the target area that the person in the vehicle is looking at includes: Determine the target person who sent the voice message; In response to determining that the target person is looking at the target area, it is determined that the person in the vehicle is looking at the target area.

4. The method according to any one of claims 1 to 3, characterized in that: The amplification factor is a first amplification factor, a second amplification factor or a third amplification factor; Then, in response to determining that the person in the vehicle is looking at the target area and the number of the people in the vehicle is 1, determining the magnification factor includes: In response to determining that the person in the vehicle is looking at the target area, and the number of people in the vehicle is 1, determining the first magnification factor; or, In response to determining that the person in the vehicle is looking at the target area, determining the magnification factor includes: In response to determining that the person in the vehicle is looking at the target area, determining the second magnification factor; or, In response to determining that the number of people in the vehicle is 1, determining an amplification factor includes: In response to determining that the number of occupants of the vehicle is 1, the third amplification factor is determined.

5. The method according to claim 4, characterized in that In response to determining that the person in the vehicle is looking at the target area, determining the second magnification factor includes: In response to the person in the vehicle looking at the target area, acquiring a first auxiliary factor; The first auxiliary factor is determined as the second amplification factor.

6. The method according to claim 5, characterized in that After obtaining the first auxiliary factor, the method further includes: Determine whether the number of people in the vehicle is 1; if so, obtain a second auxiliary factor, and determine that the product of the first auxiliary factor and the second auxiliary factor is the second amplification factor; If not, a third auxiliary factor is obtained, and a product of the first auxiliary factor and the third auxiliary factor is determined as the second amplification factor.

7. The method according to claim 1, characterized in that After determining the first probability that the voice message is an instruction, the method further includes: In response to determining that the person in the vehicle is not looking at the target area, and the number of the people in the vehicle is greater than 1, determining a reduction factor, and correcting the first probability based on the reduction factor to obtain the second probability; Wherein, the reduction factor is less than 1.

8. The method according to any one of claims 1-3, 5-7, characterized in that: The text classification model is trained by the following method: Acquire a training sample set, and input the training sample set into a model to be trained; wherein the training sample set includes a text sample corresponding to a speech sample, and a preset label corresponding to the text sample, and the preset label includes a first label corresponding to the instruction, and a second label corresponding to a non-instruction; The parameters in the model to be trained are adjusted using the loss function in the model to be trained until the value of the loss function is less than a fourth preset threshold, thereby obtaining the text classification model; wherein the loss function includes a weight factor greater than 1, and the weight factor is used to increase the weight of the text sample corresponding to the first label.

9. A message processing device, characterized in that: include: a classification module, configured to determine, in response to receiving a voice message, a first probability that the voice message is an instruction based on a text classification model; a correction unit, configured to determine a magnification factor in response to determining that the person in the vehicle is looking at the target area and / or the number of the person in the vehicle is 1, and to correct the first probability based on the magnification factor to obtain a second probability; wherein the magnification factor is greater than 1; The instruction unit is used to determine that the voice message is the instruction in response to the second probability being greater than a first preset threshold.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images