Voice interaction method, computer equipment and computer readable storage medium
By using pre-trained natural language processing models and prior information to process vehicle voice requests on the server side, the problem of user chatting words and sentences in vehicle voice interaction are solved, and the accuracy and user experience of voice interaction are improved .
Patent Information
- Application Number
- CN202411161722.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-08
AI Technical Summary
In the vehicle voice interaction scenario, when a user chats, saying words related to or similar to vertical commands may be mistakenly recognized as vertical commands, causing the vehicle to perform actions that the user expects, affecting the user's experience of the vehicle's voice interaction function.
By receiving voice requests forwarded by the vehicle on the server side, and using pre-trained natural language processing models and prior information for speech recognition, the natural language processing results of the voice request are determined, and the processed results are sent to the vehicle to ensure the voice request Accuracy and reduce misidentification.
It improves the accuracy of the natural language processing results of voice requests, reduces the situation where the user's voice is mistakenly recognized as a vertical command, and improves the user's experience of vehicle voice interaction functions.
Smart Images

Figure CN118918884A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of voice interaction technology, and in particular to a voice interaction method, a computer device, and a computer-readable storage medium. Background Art
[0002] In the in-vehicle voice interaction scenario, in order to ensure that the natural language processing model accurately responds to the user's voice, the model will be trained through vertical commands such as vehicle control commands, navigation commands, etc. However, when the user speaks words related to or similar to vertical commands during casual chat, the user's voice may be mistakenly recognized by the vehicle as a vertical command, causing the vehicle to perform actions beyond the user's expectations, which to some extent affects the user's experience of the vehicle's voice interaction function. Summary of the invention
[0003] The present application provides a voice interaction method, a computer device, and a computer-readable storage medium.
[0004] The present application embodiment provides a voice interaction method, which is used in a server, including:
[0005] Receive voice requests forwarded by the vehicle;
[0006] Determine a first natural language processing result corresponding to the voice request according to the voice request, the first prior information and the first natural language processing model, wherein the first natural language processing model is pre-trained using a first training set, and the first prior information is determined based on the first training set;
[0007] The first natural language processing result is sent to the vehicle to complete the voice interaction.
[0008] In the voice interaction method provided in the embodiment of the present application, the server can receive a voice request forwarded by the vehicle, and determine the first natural language processing result corresponding to the voice request based on the voice request, the first prior information and the first natural language processing model, and send the first natural language processing result to the vehicle, so that the vehicle can complete the voice interaction through the first natural language processing result.
[0009] Thus, in the implementation mode of the present application, the natural language processing of the voice request can be completed according to the first prior information and the first natural language processing model, thereby guaranteeing the accuracy of the first natural language processing result of the voice request to a certain extent. Also, since the first prior information is determined according to the first training set for training the first natural language processing model, the natural language processing of the voice request can take into account the state of the first training set, such as the sample distribution state, thereby further improving the accuracy of the first natural language processing result, and further reducing the occurrence of the user's voice being misidentified as a vertical command to a certain extent, and the user's experience of the vehicle voice interaction function is guaranteed.
[0010] In some embodiments of the present application, the samples in the first training set include multiple semantic units, and the method further includes:
[0011] Determining first occurrence frequencies of a plurality of preset text units in the first training set according to samples in the first training set, wherein the preset text units include at least two semantic units;
[0012] According to a predetermined first frequency upper limit and a first frequency lower limit, smoothing is performed on the first occurrence frequency of each of the preset text units to determine a smoothed frequency;
[0013] Determining the occurrence probability of the preset text unit according to the smoothed frequency of the preset text unit and the smoothed frequency of the target text unit corresponding to the preset text unit, wherein a semantic unit in the preset text unit is the same as a semantic unit in the target text unit;
[0014] The first priori information is determined according to the occurrence probability of each of the preset text units.
[0015] Thus, in an embodiment of the present application, the first occurrence frequencies of multiple preset text units in the first training set can be determined based on the samples in the first training set, and the first occurrence frequency of each preset text unit can be smoothed to determine the smoothed frequency based on the predetermined first frequency upper limit and the first frequency lower limit, and the occurrence probability of the preset text unit can be determined based on the smoothed frequency of the preset text unit and the smoothed frequency of the target text unit corresponding to the preset text unit, and then the first prior information can be determined based on the occurrence probability of each preset text unit.
[0016] In certain embodiments of the present application, determining the first natural language processing result corresponding to the voice request according to the voice request, the first prior information and the first natural language processing model includes:
[0017] Predicting a semantic unit corresponding to the voice request according to the voice request and the first natural language processing model, and determining first prediction probability information;
[0018] Determining second prediction probability information according to the first prediction probability information and the first priori information;
[0019] The first natural language processing result is determined according to the second prediction probability information.
[0020] Thus, in an embodiment of the present application, when the server predicts the semantic unit corresponding to a voice request through the first natural language model, the second prediction probability information can be determined based on the first prior information and the first prediction probability information output by the first natural language model for the voice request, and the first natural language processing result can be determined based on the second prediction probability information, so that the prediction probability of the first natural language model for the voice request can be adjusted based on the first prior information.
[0021] The present application provides a voice interaction method for a vehicle, the method comprising:
[0022] Get voice request;
[0023] When a first natural language processing result sent by a server can be received, the voice interaction is completed according to the first natural language processing result, wherein the first natural language processing result is obtained by the server processing the voice request according to the above-mentioned voice interaction method.
[0024] Thus, in the implementation mode of the present application, the natural language processing of the voice request can be completed according to the first prior information and the first natural language processing model, thereby guaranteeing the accuracy of the first natural language processing result of the voice request to a certain extent. Also, since the first prior information is determined according to the first training set for training the first natural language processing model, the natural language processing of the voice request can take into account the state of the first training set, such as the sample distribution state, thereby further improving the accuracy of the first natural language processing result, and further reducing the occurrence of the user's voice being misidentified as a vertical command to a certain extent, and the user's experience of the vehicle voice interaction function is guaranteed.
[0025] In certain embodiments of the present application, it also includes:
[0026] Determine a second natural language processing result corresponding to the voice request according to the voice request, the second prior information and the second natural language processing model, wherein the second natural language processing model is pre-trained using a second training set, and the second prior information is determined based on the second training set;
[0027] In the event that the first natural language processing result cannot be received, the voice interaction is completed according to the second natural language processing result.
[0028] In this way, in the implementation mode of the present application, the vehicle can determine the second natural language processing result corresponding to the voice request based on the voice request, the second prior information and the second natural language processing model, and then complete the voice interaction according to the second natural language processing result when the first natural language processing result cannot be received, thereby ensuring the stable operation of the voice interaction function of the vehicle to a certain extent.
[0029] In some embodiments of the present application, the samples in the second training set include multiple semantic units, and the method further includes:
[0030] Determining a second occurrence frequency of each semantic unit in the second training set according to samples in the second training set;
[0031] According to a predetermined second frequency upper limit and a second frequency lower limit, smoothing the second occurrence frequency of each semantic unit in the second training set to determine a processed frequency of each semantic unit in the second training set;
[0032] Determining the occurrence probability of each semantic unit according to the processed frequency of each semantic unit in the second training set;
[0033] The second prior information is determined according to the occurrence probability of each semantic unit in the second training set.
[0034] Thus, in the implementation mode of the present application, the second occurrence frequency of each semantic unit in the second training set can be determined based on the samples in the second training set, and the second occurrence frequency of each semantic unit in the second training set can be smoothed by a predetermined second frequency upper limit and a second frequency lower limit to determine the processed frequency of each semantic unit in the second training set, and then the occurrence probability of each semantic unit can be determined based on the processed frequency of each semantic unit in the second training set, and the second prior information can be determined based on the occurrence probability of each semantic unit in the second training set.
[0035] In certain embodiments of the present application, determining the second natural language processing result corresponding to the voice request according to the voice request, the second prior information and the second natural language processing model includes:
[0036] Predicting a semantic unit corresponding to the voice request according to the voice request and the second natural language processing model, and determining third prediction probability information;
[0037] Determining fourth prediction probability information according to the third prediction probability information and the second priori information;
[0038] The second natural language processing result is determined according to the fourth prediction probability information.
[0039] Thus, in an embodiment of the present application, when the server predicts the semantic unit corresponding to a voice request through a second natural language model, it can determine the fourth prediction probability information based on the second prior information and the third prediction probability information output by the second natural language model for the voice request, and determine the second natural language processing result based on the fourth prediction probability information.
[0040] In certain embodiments of the present application, the method further comprises:
[0041] The second prior information is updated according to the acquired historical voice requests.
[0042] In this way, in the implementation mode of the present application, the vehicle can update the second prior information according to the acquired historical voice request, so that the second prior information can match the user's personal preference indicated by the historical voice request to a certain extent.
[0043] An embodiment of the present application provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned voice interaction method is implemented.
[0044] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the above-mentioned voice interaction method is implemented.
[0045] The computer device and computer-readable storage medium provided in the embodiments of the present application enable the natural language processing of the voice request to be completed according to the first prior information and the first natural language processing model, thereby ensuring the accuracy of the first natural language processing result of the voice request to a certain extent. In addition, since the first prior information is determined according to the first training set for training the first natural language processing model, the natural language processing of the voice request can take into account the state of the first training set, such as the sample distribution state, thereby further improving the accuracy of the first natural language processing result, thereby reducing the occurrence of the user's voice being misidentified as a vertical command to a certain extent, and ensuring the user's experience of the vehicle voice interaction function.
[0046] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0048] Figure 1This is a schematic diagram of in-vehicle voice interaction;
[0049] Figure 2 A flowchart of a voice interaction method in certain implementation modes of the present application;
[0050] Figure 3 A flowchart of a voice interaction method in certain implementation modes of the present application;
[0051] Figure 4 A schematic diagram of application scenarios in certain embodiments of the present application;
[0052] Figure 5 A flowchart of a voice interaction method in certain implementation modes of the present application;
[0053] Figure 6 A schematic diagram of application scenarios in certain embodiments of the present application;
[0054] Figure 7 A schematic diagram of application scenarios in certain embodiments of the present application;
[0055] Figure 8 A flowchart of a voice interaction method in certain implementation modes of the present application;
[0056] Fig. 9 A flowchart of a voice interaction method in certain implementation modes of the present application;
[0057] Fig.10 A flowchart of a voice interaction method in certain implementation modes of the present application;
[0058] Fig.11 A schematic diagram of application scenarios in certain embodiments of the present application;
[0059] Fig.12 A flowchart of a voice interaction method in certain implementation modes of the present application;
[0060] Fig.13 A schematic diagram of application scenarios in certain embodiments of the present application;
[0061] Fig.14 A schematic diagram of an application scenario in certain embodiments of the present application. DETAILED DESCRIPTION
[0062] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.
[0063] In the in-vehicle voice interaction scenario, in order to ensure that the natural language processing model accurately responds to the user's voice, the proportion of vertical commands such as vehicle control commands and navigation commands in the training set used when training the natural language processing model may be relatively high. Among them, vehicle control commands can be understood as voice commands such as "turn on the air conditioner" and "turn off the headlights" used to control the vehicle and vehicle parts, and navigation commands can be understood as voice commands related to navigation functions such as "navigate to the restaurant" and "navigate to the company".
[0064] Furthermore, since the proportion of each vertical command in the training set is relatively high, after the training is completed, if a voice message is input to the natural language processing model, and the voice message contains words and phrases related to or similar to the aforementioned vertical commands, the model has a high probability of recognizing the voice message as a vertical command and outputting it. Therefore, when users chat in the car, the chat words and phrases may be mistakenly recognized by the vehicle as the aforementioned vertical commands, causing the vehicle to perform actions beyond the user's expectations, thereby affecting the user's driving experience.
[0065] For further information, see Figure 1 , Figure 1 It is a schematic diagram of in-vehicle voice interaction. That is, in a related technology, in an in-vehicle voice interaction scenario, when the user makes a voice, the vehicle captures the user voice to obtain the user audio, and uses the local acoustic model and voice model to recognize the user audio to obtain the local recognition text corresponding to the user audio. At the same time, the vehicle reports the user audio to the cloud server, and the cloud server recognizes the user audio reported by the vehicle according to the acoustic model and language model of the cloud to obtain the cloud recognition text corresponding to the user audio. Finally, the vehicle uses one of the local recognition text and the cloud recognition text to refresh the display content of the vehicle display components (such as the central control display) for the user to display the voice recognition effect, and for subsequent natural language processing (Natural Language Processing, NLP) and response.
[0066] It is understandable that in Figure 1 The model size of the cloud acoustic model shown is usually larger than the model size of the local acoustic model, or in other words, the prediction accuracy of the cloud acoustic model for vertical commands is higher than that of the local acoustic model. Therefore, the probability that the cloud language model recognizes the received user voice as a vertical command is higher than the probability that the local language model recognizes the received user voice as a vertical command. In other words, when voice interaction is performed based on the cloud model, the probability of the above-mentioned "vehicle performing actions beyond the user's expectations" is higher.
[0067] In related technologies, in order to prevent the vehicle from performing actions beyond the user's expectations, one solution is to collect enough audio data of small talk as training samples for the acoustic model, so that the acoustic model can distinguish between small talk between users in the car and human-vehicle voice interaction. However, acoustic audio training data is scarce and difficult to collect, and the words and sentences used in small talk are more casual, covering a wide range of fields, and the voice quality is poor. Therefore, it is difficult to collect small talk voice, resulting in high implementation difficulty and real-time cost of this solution.
[0068] Based on the above problems you may encounter, please refer to Figure 2 , the present application embodiment provides a voice interaction method for a server, comprising:
[0069] 01: Receive the voice request forwarded by the vehicle;
[0070] 02: Determine a first natural language processing result corresponding to the voice request according to the voice request, the first prior information and the first natural language processing model, wherein the first natural language processing model is pre-trained using a first training set, and the first prior information is determined based on the first training set;
[0071] 03: Send the first natural language processing result to the vehicle to complete the voice interaction.
[0072] The embodiment of the present application provides a language interaction device for a server. The voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes a receiving module, a first result determination module and a first voice interaction module. Among them, the receiving module is used to receive a voice request forwarded by a vehicle. The first result determination module is used to determine the first natural language processing result corresponding to the voice request according to the voice request, the first prior information and the first natural language processing model. The first natural language processing model is pre-trained with a first training set, and the first prior information is determined according to the first training set. The first voice interaction module is used to send the first natural language processing result to the vehicle to complete the voice interaction.
[0073] The embodiment of the present application also provides a computer device, which includes a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the computer device of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to receive a voice request forwarded by a vehicle, and to determine a first natural language processing result corresponding to the voice request based on the voice request, the first prior information and the first natural language processing model, and to send the first natural language processing result to the vehicle to complete the voice interaction, wherein the first natural language processing model is pre-trained with a first training set, and the first prior information is determined based on the first training set.
[0074] Specifically, in an embodiment of the present application, when a server (or computer device) receives a voice request forwarded by a vehicle, it may use the voice request, predetermined first prior information, and a pre-trained first natural language processing model to perform natural language processing such as voice recognition on the voice request to obtain a first natural language processing result of the voice request, and send the first natural language processing result to the vehicle, so that the vehicle can complete voice interaction with the user based on the first natural language processing result.
[0075] For example, when a user speaks a sentence and the vehicle captures the corresponding audio data, the vehicle can determine the audio data as a voice request and forward it to the server. After receiving the voice request forwarded by the vehicle, the server can combine the first prior information and the first natural language processing model to perform natural language processing such as voice recognition on the voice request, thereby determining that the text corresponding to the voice request is "turn on the air conditioner". The server can then send the text "turn on the air conditioner" to the vehicle, so that the vehicle starts the vehicle air conditioner according to the text "turn on the air conditioner", thereby completing the interaction with the user.
[0076] For another example, after receiving the voice request forwarded by the vehicle, the server combines the first prior information and the first natural language processing model to perform natural language processing such as voice recognition on the voice request, thereby determining that the text corresponding to the voice request is "Pioneer Strike!" The server can then send the text "Pioneer Strike!" to the vehicle, so that the vehicle determines that the user in the car is chatting based on the text "Pioneer Strike!" and therefore ignores the voice request and the corresponding text, thereby completing the interaction with the user.
[0077] Thus, in the implementation mode of the present application, the natural language processing of the voice request can be completed according to the first prior information and the first natural language processing model, thereby guaranteeing the accuracy of the first natural language processing result of the voice request to a certain extent. Also, since the first prior information is determined according to the first training set for training the first natural language processing model, the natural language processing of the voice request can take into account the state of the first training set, such as the sample distribution state, thereby further improving the accuracy of the first natural language processing result, and further reducing the occurrence of the user's voice being misidentified as a vertical command to a certain extent, and the user's experience of the vehicle voice interaction function is guaranteed.
[0078] It is understood that, in the implementation of the present application, the voice request can be understood as audio information or audio data. In one example, the vehicle can capture the sound emitted by the user in the cabin space through a microphone in the cabin space to obtain the voice request.
[0079] It can also be understood that in the implementation of the present application, the first natural language processing model can be set according to actual conditions. For example, in one example, the first natural language processing model is an acoustic model (AM), which is used to recognize phonemes or pronunciations of the received voice request to obtain a pronunciation prediction corresponding to the voice request.
[0080] For example, in another example, the first natural language processing model is a combination of an acoustic model and a language model (LanguageModel, LM), which is used to perform speech recognition on a received voice request to obtain a text prediction corresponding to the voice request.
[0081] It can be understood that in the implementation manner of the present application, the first natural language processing model has been pre-trained and is suitable for downstream tasks in vehicle voice scenarios, such as vehicle control command recognition, navigation command recognition, etc.
[0082] It is also understandable that since it is difficult to collect audio data, especially chat audio data, in an embodiment of the present application, the server can improve the quality of the model prediction results through predetermined first prior information, or in other words, improve the accuracy of the prediction results of the first natural language processing model for voice requests.
[0083] It should be noted that, in the implementation manner of the present application, the first prior information is determined by the first training set of the first natural language processing model, and can reflect the state of the first training set to a certain extent, such as the distribution state of the samples in the first training set.
[0084] It should also be noted that when a neural network model performs prediction work, it usually predicts the probability of an input data relative to multiple optional results. For example, when a two-class acoustic model receives an audio data input, the model can predict whether the pinyin corresponding to the audio data is "xiang1" or "xian1". Among them, '1' represents the yin level, which is the first tone.
[0085] Therefore, in one example, the server can adjust the prediction probability of the first natural language processing model for a voice request through the first prior information. For example, if the first prior information is "in the training set of the first natural language processing model, the number of semantic units pronounced as 'xiang1' is higher than the number of semantic units pronounced as 'xian1'", and thus the probability that the model predicts the pinyin corresponding to an audio data as "xiang1" may be higher than the probability that the model predicts the pinyin corresponding to an audio data as "xian1", then the server can use the first prior information to reduce the probability that the model predicts the pinyin corresponding to the audio data as "xiang1" and increase the probability that the model predicts the pinyin corresponding to the audio data as "xian1" when the model predicts whether the pinyin corresponding to the audio data is "xiang1" or "xian1", thereby avoiding the model prediction defect caused by the uneven distribution of samples in the data set, that is, "the probability that the model predicts the pinyin corresponding to an audio data as 'xiang1' may be higher than the probability that the model predicts the pinyin corresponding to the audio data as 'xian1'".
[0086] See also Figure 3 In some embodiments of the present application, the samples in the first training set include multiple text units, each text unit includes at least two semantic units, and further, the voice interaction method of the embodiment of the present application further includes:
[0087] 04: determining, based on samples in the first training set, first occurrence frequencies of a plurality of preset text units in the first training set, wherein the preset text units include at least two semantic units;
[0088] 05: According to the predetermined first frequency upper limit and the first frequency lower limit, the first occurrence frequency of each preset text unit is smoothed to determine the smoothed frequency;
[0089] 06: determining the occurrence probability of the preset text unit according to the smoothed frequency of the preset text unit and the smoothed frequency of the target text unit corresponding to the preset text unit, wherein a semantic unit in the preset text unit is the same as a semantic unit in the target text unit;
[0090] 07: Determine first prior information according to the occurrence probability of each preset text unit.
[0091] The speech interaction device of the embodiment of the present application also includes a first frequency determination module, a first frequency smoothing module, a text unit occurrence probability determination module, and a first prior information determination module. Among them, the first frequency determination module is used to determine the first occurrence frequency of multiple preset text units in the first training set based on the samples in the first training set, and the preset text unit includes at least two semantic units. The first frequency smoothing module is used to smooth the first occurrence frequency of each preset text unit according to the predetermined first frequency upper limit and the first frequency lower limit, and determine the smoothed frequency. The text unit occurrence probability determination module is used to determine the occurrence probability of the preset text unit according to the smoothed frequency of the preset text unit and the smoothed frequency of the target text unit corresponding to the preset text unit, and a semantic unit in the preset text unit is the same as a semantic unit in the target text unit. The first prior information determination module is used to determine the first prior information according to the occurrence probability of each preset text unit.
[0092] The processor of the implementation mode of the present application is also used to determine the first occurrence frequency of multiple preset text units in the first training set based on the samples in the first training set, and to smooth the first occurrence frequency of each preset text unit according to the predetermined first frequency upper limit and the first frequency lower limit to determine the smoothed frequency, and to determine the occurrence probability of the preset text unit according to the smoothed frequency of the preset text unit and the smoothed frequency of the target text unit corresponding to the preset text unit, and to determine the first prior information according to the occurrence probability of each preset text unit, the preset text unit includes at least two semantic units, a semantic unit in the preset text unit is the same as a semantic unit in the target text unit, and the text unit can be understood as being determined by a combination of two or more semantic units.
[0093] Specifically, in the implementation manner of the present application, the first prior information of the server can be determined according to the frequency of occurrence of the text unit of each sample in the first training set.
[0094] Furthermore, in the embodiments of the present application, the semantic unit can be understood as a token, such as "xian1", "xiang1", "fen1", "feng1", etc. Correspondingly, a text unit consisting of two semantic units can be understood as binary tokens, such as "xiang1 fen1", and ternary tokens, tetra-element tokens, quinary tokens, etc. can be understood in the same way.
[0095] To more clearly illustrate the implementation of this application, please refer to Figure 4 , Figure 4Schematic diagram of application scenarios in certain embodiments of the present application. That is, when the preset text unit is the above-mentioned binary tokens, and the preset text unit is a bigram, then: for a predetermined first training set, the server can first count the number of occurrences of each binary token in the first training set.
[0096] For example, suppose a training set includes two samples, one of which is "da3 kai1 kong1 tiao2" and the other is "da3 kai1 che1chuang1". In this training set, the frequency of occurrence of the binary token "da3 kai1" is 2, the frequency of occurrence of the binary token "kong1 tiao2" is 1, and the frequency of occurrence of the binary token "che1chuang1" is 1. Among them, '2' represents Yangping, i.e. the second tone, and '3' represents Shangsheng, i.e. the third tone.
[0097] Next, the server may determine the number of occurrences of each bigram in the D*D bigrams in the first training set according to the number of occurrences of each binary token in the first training set.
[0098] For example, Figure 4 As shown in the figure, according to the number of occurrences of bigrams such as "xiang1 fen1" and "xian1feng1" in all samples included in a training set, the number of bigrams "xiang1 fen1" is assigned to 1,000,000, and the number of bigrams "xian1 feng1" is assigned to 20,000. At the same time, since the bigram "xiang1 feng1" is not included in all samples included in the training set, the number of bigrams "xiang1 feng1" is assigned to 0.
[0099] Subsequently, after counting the number of occurrences of each bigram in the D*D bigrams in the first training set, the server may smooth the occurrence frequency of each bigram according to a predetermined first frequency upper limit and a first frequency lower limit to determine the smoothed frequency of each bigram.
[0100] For example, Figure 4 As shown, the first frequency upper limit (max) is 500000, and the first frequency lower limit (min) is 5, so the occurrence frequency of the bigram "xiang1 fen1" is smoothed to 500000, the occurrence frequency of the bigram "xian1 feng1" is maintained at 20000, and the occurrence frequency of the bigram "xiang1 feng1" is smoothed to 5.
[0101] Finally, the server may calculate the occurrence probability of each bigram according to the smoothed frequency of each bigram, thereby obtaining the first prior information.
[0102] For example, as shown in Figure 4, the occurrence probability of "xiang1 fen1" is 0.1, the occurrence probability of xian1 feng1 is 0.01, and the occurrence probability of xiang1 feng1 is 0.0001.
[0103] Optionally, let a bigram be "xian1 feng1", then the probability of occurrence of "xian1 feng1" can be calculated by the following formula, namely:
[0104]
[0105] Wherein, B(xian1,feng1) represents the occurrence probability of “xian1 feng1”, C(xian1,feng1) represents the smoothed frequency of “xian1 feng1”, and C represents the smoothed frequency of the target binary tokens (or target text unit) corresponding to “xian1 feng1”, which refers to all the binary tokens including “xian1” among the D*D binary tokens.
[0106] Furthermore, if Figure 4 As shown, the first prior information may be the occurrence probability B of each bigram in the D*D bigrams, or in other words, the occurrence probability B of each bigram in the D*D bigrams is used to form a probability matrix to obtain the first prior information.
[0107] It can be understood that in the embodiment of the present application, since the first occurrence frequencies of multiple preset text units in the first training set are determined by samples in the first training set, the first occurrence frequencies of the preset text units can reflect the distribution of the preset text units in the first training set.
[0108] It can also be understood that, since the first occurrence frequency of the preset text unit is smoothed by the preset first frequency upper limit and the first frequency lower limit, the first occurrence frequency of the preset text unit with a higher first occurrence frequency can be suppressed, while the first occurrence frequency of the preset text unit with a lower first occurrence frequency or even a preset text unit that has not appeared in the first training set can be stimulated.
[0109] For example, in the above example, the first occurrence frequency of "xiang1 fen1" is 1000000, and the first occurrence frequency of "xiang1feng1" is 0. After smoothing, the smoothed frequency of "xiang1 fen1" is 500000, and the smoothed frequency of "xiang1feng1" is 5. In other words, the occurrence frequency of "xiang1 fen1" is suppressed, and the occurrence frequency of "xiang1 feng1" is stimulated.
[0110] Furthermore, after completing the smoothing processing of each preset text unit, the occurrence probability corresponding to the preset text unit can be calculated to obtain the first prior information. Then, when the first natural language processing model predicts the text unit corresponding to a voice request, the server can adjust the prediction probability of the first natural language processing model for each text unit based on the first prior information, so as to increase the probability that the first natural language processing model predicts that the "text unit corresponding to the voice request" is the "preset text unit with a lower probability of occurrence", and reduce the probability that the first natural language processing model predicts that the "text unit corresponding to the voice request" is the "preset text unit with a higher probability of occurrence", thereby weakening the negative impact caused by the uneven distribution of samples in the first training set.
[0111] Thus, in an embodiment of the present application, the first occurrence frequencies of multiple preset text units in the first training set can be determined based on the samples in the first training set, and the first occurrence frequency of each preset text unit can be smoothed to determine the smoothed frequency based on the predetermined first frequency upper limit and the first frequency lower limit, and the occurrence probability of the preset text unit can be determined based on the smoothed frequency of the preset text unit and the smoothed frequency of the target text unit corresponding to the preset text unit, and then the first prior information can be determined based on the occurrence probability of each preset text unit.
[0112] In addition, in Figure 4 In some embodiments shown in the figure, in order to further improve the incentive effect of the preset text unit with a low first appearance frequency or even not appearing in the first training set, the server can set the appearance frequency of the preset text unit to a default value for the preset text unit with too low appearance frequency to achieve probability smoothing. The default value can be as follows: Figure 4 Therefore, the probability of "xiang1 feng1" originally was 0.0001. After smoothing, the probability of "xiang1 feng1" increased to 0.001.
[0113] See also Figure 5 In certain embodiments of the present application, step 02 includes:
[0114] 020: predicting a semantic unit corresponding to the voice request according to the voice request and the first natural language processing model, and determining first prediction probability information;
[0115] 021: Determine second prediction probability information according to the first prediction probability information and the first priori information;
[0116] 022: Determine the first natural language processing result according to the second prediction probability information.
[0117] The first result determination module of the implementation mode of the present application is also used to predict the semantic unit corresponding to the voice request based on the voice request and the first natural language processing model, determine the first prediction probability information, and to determine the second prediction probability information based on the first prediction probability information and the first prior information, and to determine the first natural language processing result based on the second prediction probability information.
[0118] The processor of the embodiment of the present application is also used to predict the semantic unit corresponding to the voice request based on the voice request and the first natural language processing model, determine the first prediction probability information, and to determine the second prediction probability information based on the first prediction probability information and the first prior information, and to determine the first natural language processing result based on the second prediction probability information.
[0119] Specifically, in an embodiment of the present application, when the server obtains the first prior information, it may use the first prior information to adjust the first prediction probability information output by the first natural language processing model for the voice request to obtain the second prediction probability information, and then predict the semantic unit corresponding to the voice request based on the second prediction probability information, thereby completing the natural language processing of the voice request.
[0120] To more clearly illustrate the implementation of this application, please refer to Figure 6 and Figure 7 , Figure 6 and Figure 7 All of them are schematic diagrams of application scenarios in certain embodiments of the present application. That is, the first natural language processing model includes an encoder and a decoder, the encoder is composed of N Conformer modules, and the decoder is composed of M Transformer modules.
[0121] Furthermore, the softmax of the decoder at each moment can output N*D probability values, where N represents the number of beams, i.e., Y at the previous moment tThe N candidate results are retained and then fed into the decoder based on the N candidate results to generate N softmax output probabilities, or N first prediction probabilities. Then, each softmax output probability is removed by the first prior information to determine the new softmax output probability, which is the second prediction probability in the embodiment of the present application. The process of generating the second prediction probability can be expressed by the following formula, namely
[0122]
[0123] Where N is Y t The number of . i represents the i-th output node of softmax. Represents the autoregressive input Y at time t t The new softmax output probability is the second prediction probability in the embodiment of the present application. N,i Represents the autoregressive input Y at time t t The output probability of softmax is the first prediction probability in the implementation mode of this application. t ) is the autoregressive input Y at time t t When (Y t ,i) is the bigram prior probability, that is, the first prior information related to Y t The corresponding probability information.
[0124] Therefore, if Figure 6 As shown, without using the first prior information, the server predicts the semantic units of the received voice request through the first natural language processing model, and predicts the semantic unit sequence of "dao4, di3, shei2, shi4, xiang1, fen1", thereby obtaining the natural language text "Who is the fragrance". Among them, '4' represents the falling tone, that is, the fourth tone.
[0125] And as Figure 4As shown, the server predicts the semantic unit of the received voice request through the first natural language processing model, and when it is determined that the prediction probability of the semantic unit "xiang1" corresponding to the voice request is 0.7 and the prediction probability of the semantic unit "xian1" corresponding to the voice request is 0.2, the two probabilities are adjusted based on the first prior information so that the prediction probability of the semantic unit "xiang1" is 0.4 and the prediction probability of the semantic unit "xian1" is 0.5. And, when it is determined that the prediction probability of the semantic unit "feng1" corresponding to the voice request is 0.4 and the prediction probability of the semantic unit "fen1" corresponding to the voice request is 0.5, the two probabilities are adjusted based on the first prior information so that the prediction probability of the semantic unit "feng1" is 0.55 and the prediction probability of the semantic unit "fen1" is 0.3. Thus, the server can obtain the semantic unit sequence "dao4, di3, shei2, shi4, xian1, feng1", and determine the natural language text "Who is the pioneer" through this semantic unit sequence.
[0126] In one example, after obtaining the second prediction probability information, the vehicle can determine the semantic unit sequence corresponding to the voice request, such as the above-mentioned "dao4, di3, shei2, shi4, xian1, feng1". Then the semantic unit sequence is input into a pre-trained language model (LM) to convert the semantic unit sequence into natural language text through the language model, such as the above-mentioned "Who is the pioneer", thereby obtaining the first natural language processing result.
[0127] Alternatively, according to a predetermined decoding graph, the semantic unit sequence is mapped into a natural language text similar to the above-mentioned "Who is the pioneer", thereby obtaining a first natural language processing result.
[0128] Thus, in an embodiment of the present application, when the server predicts the semantic unit corresponding to a voice request through the first natural language model, the second prediction probability information can be determined based on the first prior information and the first prediction probability information output by the first natural language model for the voice request, and the first natural language processing result can be determined based on the second prediction probability information, so that the prediction probability of the first natural language model for the voice request can be adjusted based on the first prior information.
[0129] In addition, it should be noted that in Figure 4 , 6In the examples provided in 7, the first natural language processing model is an end-to-end model based on autoregression. In other words, when the first natural language processing model makes predictions, the prediction of the subsequent semantic unit is strongly dependent on the prediction of the previous semantic unit. Therefore, the binary prior probability is used as the first prior information, thereby ensuring the validity of the first prior information.
[0130] Moreover, since it is a binary prior probability, the storage cost is relatively high for the D*D binary semantic unit groups, or in other words, for the above binary tokens, the above bigrams, and the text units of the implementation mode of the present application. For example, if D is 2000, then D*D is 4 million, and the cost of storing 4 million pieces of data on the vehicle side is relatively high. Therefore, in order to ensure the robust use of the first prior information, in the implementation mode of the present application, the server completes the natural language processing of the voice request through the first prior information and the first natural language processing model.
[0131] See also Figure 8 Corresponding to the above-mentioned voice interaction method for a server, the embodiment of the present application further provides a voice interaction method for a vehicle, comprising:
[0132] 21: Get voice request;
[0133] 22: When a first natural language processing result sent by the server is received, voice interaction is completed according to the first natural language processing result, wherein the first natural language processing result is obtained by the server processing the voice request according to the above-mentioned voice interaction method for the server.
[0134] The embodiment of the present application provides a language interaction device for a vehicle. The voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes an acquisition module and a second voice interaction module. Among them, the acquisition module is used to obtain a voice request. The second voice interaction module is used to complete the voice interaction according to the first natural language processing result when it can receive the first natural language processing result sent by the server. The first natural language processing result is obtained by the server processing the voice request according to the above-mentioned voice interaction method for the server.
[0135] The embodiment of the present application also provides a computer device, which includes a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the computer device of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to obtain a voice request, and to complete the voice interaction according to the first natural language processing result when the first natural language processing result sent by the server can be received, wherein the first natural language processing result is obtained by the server processing the voice request according to the above-mentioned voice interaction method for the server.
[0136] Specifically, in the embodiment of the present application, the vehicle can capture the sound made by the user in the cockpit space through a sound capture unit such as a microphone in the cockpit space to form a voice request, and forward the voice request to the server.
[0137] Correspondingly, when the server receives a voice request forwarded by the vehicle and performs natural language processing on the voice request in combination with the first prior information and the first natural language processing model to obtain a corresponding first natural language processing result, the server may send the first natural language processing result to the vehicle.
[0138] Furthermore, when the vehicle receives the first natural language processing result sent by the server, the vehicle can perform corresponding actions according to the first natural language processing result, thereby completing voice interaction with the user.
[0139] For example, after receiving the voice request forwarded by the vehicle, the server performs natural language processing such as speech recognition on the voice request in combination with the first prior information and the first natural language processing model, thereby determining that the text corresponding to the voice request is "turn on the air conditioner". The server can then send the text "turn on the air conditioner" to the vehicle, so that the vehicle starts the vehicle air conditioner according to the text "turn on the air conditioner", thereby completing the interaction with the user.
[0140] For another example, after receiving the voice request forwarded by the vehicle, the server combines the first prior information and the first natural language processing model to perform natural language processing such as voice recognition on the voice request, thereby determining that the text corresponding to the voice request is "Pioneer Strike!" The server can then send the text "Pioneer Strike!" to the vehicle, so that the vehicle determines that the user in the car is chatting based on the text "Pioneer Strike!" and therefore ignores the voice request and the corresponding text, thereby completing the interaction with the user.
[0141] Thus, in the implementation mode of the present application, the natural language processing of the voice request can be completed according to the first prior information and the first natural language processing model, thereby guaranteeing the accuracy of the first natural language processing result of the voice request to a certain extent. Also, since the first prior information is determined according to the first training set for training the first natural language processing model, the natural language processing of the voice request can take into account the state of the first training set, such as the sample distribution state, thereby further improving the accuracy of the first natural language processing result, and further reducing the occurrence of the user's voice being misidentified as a vertical command to a certain extent, and the user's experience of the vehicle voice interaction function is guaranteed.
[0142] In addition, it can also be understood that the method for obtaining the first natural language understanding processing result can be found in the aforementioned content, and to avoid repetition, it will not be repeated here.
[0143] See also Fig. 9 In some embodiments of the present application, the voice interaction method further includes:
[0144] 23: Determine a second natural language processing result corresponding to the voice request according to the voice request, the second prior information, and the second natural language processing model, wherein the second natural language processing model is pre-trained using a second training set, and the second prior information is determined based on the second training set;
[0145] 24: When the first natural language processing result is not received, complete the voice interaction according to the second natural language processing result.
[0146] The voice interaction device of the embodiment of the present application also includes a second result determination module and a third voice interaction module. The second result determination module is used to determine the second natural language processing result corresponding to the voice request according to the voice request, the second prior information and the second natural language processing model, the second natural language processing model is pre-trained by the second training set, and the second prior information is determined according to the second training set. The third voice interaction module is used to complete the voice interaction according to the second natural language processing result when the first natural language processing result cannot be received.
[0147] The processor of the embodiment of the present application is also used to determine the second natural language processing result corresponding to the voice request based on the voice request, the second prior information and the second natural language processing model, and to complete the voice interaction according to the second natural language processing result when the first natural language processing result cannot be received, wherein the second natural language processing model is pre-trained through a second training set, and the second prior information is determined based on the second training set.
[0148] Specifically, taking into account the situation where the vehicle and the cloud server have difficulty interacting normally due to network communication anomalies, and in order to achieve redundancy of the in-vehicle voice interaction function, the vehicle can, after obtaining the voice request, perform natural language processing such as voice recognition on the voice request according to the voice request, predetermined second prior information and a pre-trained second natural language processing model to obtain a corresponding second natural language processing result.
[0149] Therefore, if the vehicle is unable to receive the first natural language processing result sent by the server due to network communication abnormalities or other unexpected situations, the vehicle can complete the subsequent voice interaction process based on the local second natural language processing result.
[0150] For example, when the second natural language processing result is a text recognition result corresponding to a voice request, the vehicle can refresh the display content of the vehicle display components (such as the central control screen) according to the text recognition result to let the user know the text recognized by the vehicle based on the user's voice.
[0151] For another example, when the second natural language processing result is a text recognition result corresponding to a voice request, the vehicle can determine the user's control intention and control instructions for the vehicle based on the text recognition result, and then perform the corresponding action. For example, when the text recognition result is "turn on the fragrance", the vehicle can turn on the car fragrance based on the text recognition result, and when the text recognition result is "Who is Pioneer?", the vehicle can ignore the text recognition result and wait for the user's next round of dialogue.
[0152] In addition, it can also be understood that in the implementation of the present application, the second natural language processing model can be set according to actual conditions. For example, in one example, the second natural language processing model is an acoustic model (AM), which is used to recognize phonemes or pronunciations of the received voice request to obtain a pronunciation prediction corresponding to the voice request.
[0153] For example, in another example, the second natural language processing model is a combination of an acoustic model and a language model (LanguageModel, LM), which is used to perform voice recognition on the received voice request to obtain the text corresponding to the voice request.
[0154] It can be understood that in the implementation mode of the present application, the second natural language processing model has been pre-trained and is suitable for downstream tasks in vehicle voice scenarios, such as vehicle control command recognition, navigation command recognition, etc.
[0155] It is also understandable that since it is difficult to collect audio data, especially chat audio data, in an embodiment of the present application, the server can improve the quality of the model prediction results through predetermined second prior information, or in other words, improve the accuracy of the prediction results of the second natural language processing model for voice requests.
[0156] It should be noted that in the implementation manner of the present application, the second prior information is determined by the second training set of the second natural language processing model, and can reflect the state of the second training set to a certain extent, such as the distribution state of the samples in the first training set.
[0157] It should also be noted that when a neural network model performs prediction work, it usually predicts the probability of an input data relative to multiple optional results. For example, when a two-class acoustic model receives an audio data input, the model can predict whether the pinyin corresponding to the audio data is "xiang1" or "xian1". Among them, '1' represents the yin level, which is the first tone.
[0158] Therefore, in one example, the server can adjust the prediction probability of the second natural language processing model for a voice request through the second prior information. For example, if the second prior information is "in the training set of the second natural language processing model, the number of semantic units pronounced as 'xiang1' is higher than the number of semantic units pronounced as 'xian1'", and thus the probability that the model predicts the pinyin corresponding to an audio data as "xiang1" may be higher than the probability that the model predicts the pinyin corresponding to an audio data as "xian1", then the server can use the second prior information to reduce the probability that the model predicts the pinyin corresponding to the audio data as "xiang1" and increase the probability that the model predicts the pinyin corresponding to the audio data as "xian1" when the model predicts whether the pinyin corresponding to the audio data is "xiang1" or "xian1", thereby avoiding the model prediction defect caused by the uneven distribution of samples in the data set, that is, the situation that "the probability that the model predicts the pinyin corresponding to an audio data as 'xiang1' may be higher than the probability that the model predicts the pinyin corresponding to the audio data as 'xian1'".
[0159] In addition, in one example, the second prior information is the aforementioned first prior information, and the second natural language processing model is the aforementioned first natural language processing model.
[0160] In this way, in the implementation mode of the present application, the vehicle can determine the second natural language processing result corresponding to the voice request based on the voice request, the second prior information and the second natural language processing model, and then complete the voice interaction according to the second natural language processing result when the first natural language processing result cannot be received, thereby ensuring the stable operation of the voice interaction function of the vehicle to a certain extent.
[0161] See also Fig.10 In some implementations of the present application, the samples in the second training set include multiple semantic units. Furthermore, the voice interaction method of the implementation of the present application further includes:
[0162] 25: determining the second occurrence frequency of each semantic unit in the second training set according to the samples in the second training set;
[0163] 26: performing smoothing processing on the second occurrence frequency of each semantic unit in the second training set according to the predetermined second frequency upper limit and the second frequency lower limit, and determining a processed frequency of each semantic unit in the second training set;
[0164] 27: Determine the occurrence probability of each semantic unit according to the processed frequency of each semantic unit in the second training set;
[0165] 28: Determine the second prior information according to the occurrence probability of each semantic unit in the second training set.
[0166] The voice interaction device of the embodiment of the present application also includes a second frequency determination module, a second frequency smoothing module, a semantic unit occurrence probability determination module and a second prior information determination module. Among them, the second frequency determination module is used to determine the second occurrence frequency of each semantic unit in the second training set based on the samples in the second training set. The second frequency smoothing module is used to smooth the second occurrence frequency of each semantic unit in the second training set according to a predetermined second frequency upper limit and a second frequency lower limit, and determine the processed frequency of each semantic unit in the second training set. The semantic unit occurrence probability determination module is used to smooth the second occurrence frequency of each semantic unit in the second training set according to a predetermined second frequency upper limit and a second frequency lower limit, and determine the processed frequency of each semantic unit in the second training set. The second prior information determination module is used to determine the second prior information according to the processed frequency of each semantic unit in the second training set.
[0167] The processor of the embodiment of the present application is also used to determine the second occurrence frequency of each semantic unit in the second training set based on the samples in the second training set, and to smooth the second occurrence frequency of each semantic unit in the second training set based on a predetermined second frequency upper limit and a second frequency lower limit to determine the processed frequency of each semantic unit in the second training set, and to smooth the second occurrence frequency of each semantic unit in the second training set based on a predetermined second frequency upper limit and a second frequency lower limit to determine the processed frequency of each semantic unit in the second training set, and to determine the second prior information based on the processed frequency of each semantic unit in the second training set.
[0168] Specifically, in the implementation manner of the present application, the second prior information can be determined according to the occurrence frequency of each semantic unit in the second training set.
[0169] To more clearly illustrate the implementation of this application, please refer to Fig.11 , Fig.11 This is a schematic diagram of an application scenario in some embodiments of the present application. That is, assuming that a single semantic unit is a unigram, then for a predetermined second training set, the server may first count the number of occurrences of each unigram in the second training set.
[0170] For example, Fig.11 As shown, the number of occurrences of each unigram unit in all samples included in a second training set is counted, and then: the number of occurrences of the unigram "xiang1" in the second training set is 1500000, the number of occurrences of the unigram "xian1" is 100000, the number of occurrences of the unigram "fen1" is 60000, the number of occurrences of the unigram "feng1" is 20000,…, and the number of occurrences of the unigram "di2" is 1.
[0171] Subsequently, after determining the number of occurrences of each unigram in the second training set, the vehicle may smooth the occurrence frequency of each unigram using a predetermined second frequency upper limit and a second frequency lower limit to determine a smoothed frequency of each unigram.
[0172] For example, Fig.11As shown, the second frequency upper limit (max) is 500000, and the second frequency lower limit (min) is 5, so the number of occurrences of the unigram "xiang1" is 500000, the number of occurrences of the unigram "xian1" is maintained at 100000, the number of occurrences of the unigram "fen1" is maintained at 60000, the number of occurrences of the unigram "feng1" is maintained at 20000,..., and the number of occurrences of the unigram "di2" is smoothed to 5.
[0173] Finally, the server may calculate the occurrence probability of each unigram according to the smoothed frequency of each unigram, thereby obtaining the second prior information.
[0174] For example, as shown in Figure 4, the probability of occurrence of the unigram "xiang1" is 0.071, the probability of occurrence of the unigram "xian1" is 0.032, the probability of occurrence of the unigram "fen1" is 0.024, the probability of occurrence of the unigram "feng1" is 0.014, ..., and the probability of occurrence of the unigram "di2" is 2.2e-4.
[0175] In one example, the probability of a unigram occurrence can be calculated by the following formula:
[0176]
[0177] In the formula, B i represents the probability of occurrence of the i-th unigram. i represents the smoothed frequency of the i-th unigram.
[0178] It can be understood that in the embodiment of the present application, since the first occurrence frequencies of multiple preset text units in the first training set are determined by samples in the first training set, the first occurrence frequencies of the preset text units can reflect the distribution of the preset text units in the first training set.
[0179] It can also be understood that, because the second occurrence frequency of each semantic unit in the second training set is smoothed by the preset second frequency upper limit and second frequency lower limit, the second occurrence frequency of the semantic unit with a higher second occurrence frequency can be suppressed, while the second occurrence frequency of the semantic unit with a lower second occurrence frequency can be stimulated.
[0180] For example, in the above example, the second occurrence frequency of "xiang1" is 1500000, and the second occurrence frequency of "di2" is 1. After smoothing, the smoothed frequency of "xiang1" is 500000, and the smoothed frequency of "di2" is 5. In other words, the occurrence frequency of "xiang1" is suppressed, and the occurrence frequency of "di2" is stimulated.
[0181] Furthermore, after completing the smoothing processing of each semantic unit, the occurrence probability corresponding to the semantic unit can be calculated to obtain the second prior information. Then, when the second natural language processing model predicts the text unit corresponding to a voice request, the server can adjust the prediction probability of the second natural language processing model for each text unit based on the second prior information, so as to increase the probability that the second natural language processing model predicts that the "semantic unit corresponding to the voice request" is a "semantic unit with a lower probability of occurrence", and reduce the probability that the second natural language processing model predicts that the "semantic unit corresponding to the voice request" is a "semantic unit with a higher probability of occurrence", thereby weakening the negative impact caused by the uneven distribution of samples in the first training set.
[0182] In one example, when the vehicle uses the second natural language processing model to recognize the voice request to identify the third prediction probabilities of multiple semantic units, the vehicle can subtract the "probability of occurrence of the semantic unit in the second prior information" from the "third prediction probability of the semantic unit" for each semantic unit predicted by the second natural language processing model to obtain the fourth probability of the semantic unit.
[0183] It should be noted that for semantic units with lower second occurrence frequency and probability, the difference between the third prediction probability and the fourth prediction probability is smaller. However, for semantic units with higher second occurrence frequency and probability, the difference between the third prediction probability and the fourth prediction probability is larger. Therefore, the fourth prediction probability of low-frequency semantic units can be stimulated to a certain extent after normalization, while the fourth prediction of high-frequency semantic units can be suppressed to a certain extent after normalization.
[0184] Thus, in the implementation mode of the present application, the second occurrence frequency of each semantic unit in the second training set can be determined based on the samples in the second training set, and the second occurrence frequency of each semantic unit in the second training set can be smoothed by a predetermined second frequency upper limit and a second frequency lower limit to determine the processed frequency of each semantic unit in the second training set, and then the occurrence probability of each semantic unit can be determined based on the processed frequency of each semantic unit in the second training set, and the second prior information can be determined based on the occurrence probability of each semantic unit in the second training set.
[0185] See also Fig.12 In certain embodiments of the present application, step 23 includes:
[0186] 230: predicting a semantic unit corresponding to the voice request according to the voice request and the second natural language processing model, and determining third prediction probability information;
[0187] 231: Determine fourth prediction probability information according to the third prediction probability information and the second priori information;
[0188] 232: Determine a second natural language processing result according to the fourth prediction probability information.
[0189] The second result determination module of the implementation mode of the present application is also used to predict the semantic unit corresponding to the voice request based on the voice request and the second natural language processing model, determine the third prediction probability information, and to determine the fourth prediction probability information based on the third prediction probability information and the second prior information, and to determine the second natural language processing result based on the fourth prediction probability information.
[0190] The processor of the embodiment of the present application is also used to predict the semantic unit corresponding to the voice request based on the voice request and the second natural language processing model, determine the third prediction probability information, and to determine the fourth prediction probability information based on the third prediction probability information and the second prior information, and to determine the second natural language processing result based on the fourth prediction probability information.
[0191] Specifically, in an embodiment of the present application, the vehicle may adjust the third prediction probability output by the second natural language processing model for the voice request based on the second prior information to obtain a fourth prediction probability, and determine the second natural language processing result of the voice request through the fourth prediction probability.
[0192] To more clearly illustrate the implementation method of this application, please refer to Fig.11 , and see Fig.13 , Fig.13 Schematic diagram of application scenarios in certain embodiments of the present application. Specifically, Fig.11 As shown, in an example without using the second prior information, the vehicle calls the second natural language processing model to recognize a voice request.
[0193] During the process of recognizing the voice request, the second natural language model determines that a certain semantic unit corresponding to the voice request (let the unit be token_A) has a probability of 0.6 for "xiang1" and a probability of 0.3 for "xian1", or in other words, the softmax probability of token_A corresponding to "xiang1" is 0.6 and the softmax probability of "xian1" is 0.3. In addition, the second natural language model determines that another semantic unit corresponding to the voice request (let the unit be token_B) has a probability of 0.4 for "fen1" and a probability of 0.5 for "feng1", or in other words, the softmax probability of token_B corresponding to "fen1" is 0.5 and the softmax probability of "feng1" is 0.4.
[0194] Furthermore, the softmax probability of token_A corresponding to "xiang1" is higher than the softmax probability of token_A corresponding to "xian1", so token_A can be determined as "xiang1". Similarly, the softmax probability of token_B corresponding to "fen1" is higher than the softmax probability of token_B corresponding to "feng1", so token_B can be determined as "fen1", and the final semantic unit sequence is "dao4, di3, shei2, shi4, xiang1, fen1", and the mapped natural language text is "Who is the fragrance".
[0195] In contrast, Fig.13 As shown, in an example using the second prior information, the vehicle calls the second natural language processing model to recognize a voice request, and during the process of the second natural language model recognizing the voice request and outputting the softmax probability of token_A and the softmax probability of token_B, the vehicle can use the second prior information to adjust the softmax probability of token_A and the softmax probability of token_B respectively, so that the softmax probability of token_A corresponding to "xiang1" is 0.3, the softmax probability corresponding to "xian1" is 0.5, and the softmax probability of token_B corresponding to "fen1" is 0.4, and the softmax probability corresponding to "feng1" is 0.55.
[0196] Therefore, the softmax probability of token_A corresponding to "xiang1" is lower than the softmax probability of token_A corresponding to "xian1", so token_A can be determined as "xian1". Similarly, the softmax probability of token_B corresponding to "fen1" is lower than the softmax probability of token_B corresponding to "feng1", so token_B can be determined as "feng1", and the final semantic unit sequence is "dao4, di3, shei2, shi4, xian1, feng1", and the mapped natural language text is "Who is the pioneer".
[0197] In one example, the process of adjusting the softmax probability by the second prior information can refer to the following formula, or the process of adjusting the third prediction probability information by the second prior information to obtain the fourth prediction probability information can refer to the following formula, that is:
[0198] P i_new =P i / B i
[0199] Where P i_new represents the smoothed probability of the i output nodes, that is, the fourth prediction probability information in the implementation mode of the present application. i represents the pre-smoothing probability of the i output nodes, that is, the third prediction probability information in the implementation mode of the present application. i Represents the prior probability value corresponding to the i output nodes in the second prior information.
[0200] In addition, in one example, after obtaining the fourth prediction probability information, the vehicle can determine the semantic unit sequence corresponding to the voice request, such as the above-mentioned "dao4, di3, shei2, shi4, xian1, feng1". Then the semantic unit sequence is input into a pre-trained language model (LM) to convert the semantic unit sequence into natural language text through the language model, such as the above-mentioned "Who is the pioneer", thereby obtaining the second natural language processing result.
[0201] Alternatively, according to a predetermined decoding graph, the semantic unit sequence is mapped into a natural language text similar to the above-mentioned "Who is the pioneer" to obtain a second natural language processing result.
[0202] Thus, in an embodiment of the present application, when the server predicts the semantic unit corresponding to a voice request through a second natural language model, it can determine the fourth prediction probability information based on the second prior information and the third prediction probability information output by the second natural language model for the voice request, and determine the second natural language processing result based on the fourth prediction probability information.
[0203] In some embodiments of the present application, the voice interaction method further includes:
[0204] The second prior information is updated according to the acquired historical voice requests.
[0205] The voice interaction device of the embodiment of the present application further includes an updating module. The updating module is used to update the second prior information according to the acquired historical voice request.
[0206] The processor of the implementation manner of the present application is also used to update the second prior information according to the acquired historical voice request.
[0207] Specifically, in order to achieve user customization to meet the user's personal usage preferences, in the implementation mode of the present application, the vehicle can update the second prior information based on the historical language interaction instructions obtained from the vehicle, so that the second prior information matches the personal preferences of the user of the vehicle.
[0208] To more clearly illustrate the implementation of this application, please refer to Fig.14 , Fig.14 Schematic diagram of application scenarios in certain embodiments of the present application. That is, in the embodiments of the present application, the vehicle can record the user's voice request to obtain historical voice requests, and under certain conditions, such as when the recording time is greater than a certain time (such as one week), determine the occurrence frequency and probability of each semantic unit in the historical voice request based on the collected historical voice requests, and then merge the "occurrence probability of each semantic unit in the historical voice request" with the "occurrence probability of each semantic unit in the second prior information" to complete the update of the second prior information.
[0209] Furthermore, when the voice request at the current moment is processed through the second natural language processing model, the third prediction probability information output by the model can be adjusted through the updated second prior information, so that the third prediction probability information is adjusted to the fourth prediction probability information that "fits the user's semantic unit usage preference". Furthermore, the second natural language processing result determined by the fourth prediction probability information can meet the user's personal preferences and thus has a higher accuracy rate.
[0210] In this way, in the implementation mode of the present application, the vehicle can update the second prior information according to the acquired historical voice request, so that the second prior information can match the user's personal preference indicated by the historical voice request to a certain extent.
[0211] In addition, it should be noted that Fig.11 and Fig.13In the corresponding implementation manner, the second natural language processing model can be understood as a CTC (Connectionist temporal classification) model.
[0212] Furthermore, in the process of CTC model predicting voice requests, the output at the current moment only depends on the network input at the current moment, and does not depend on the output at the previous moment, that is, Y(t+1) and Y(t) are independent of each other and have no historical interdependence. Therefore, after determining the prior information through the frequency and probability of occurrence of semantic units in the second training set, the prior information is used to smooth the third prediction probability, so as to achieve a balance between high and low frequencies and units in the training data.
[0213] In other words, after the corresponding occurrence probability (or prior probability) is subtracted from the third prediction probability of each semantic unit, the third prediction probability of the high-frequency semantic unit is suppressed because the corresponding occurrence probability of the high-frequency semantic unit is larger. Conversely, since the occurrence probability of the low-frequency semantic unit is smaller, the third prediction probability of the low-frequency semantic unit can be stimulated to a certain extent compared with the high-frequency semantic unit, thereby reducing the difference in prediction probability with the high-frequency semantic unit.
[0214] An embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the above-mentioned voice interaction method is implemented.
[0215] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.
[0216] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0217] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A voice interaction method, characterized in that: For servers, including: Receive voice requests forwarded by the vehicle; Determine a first natural language processing result corresponding to the voice request according to the voice request, the first prior information and the first natural language processing model, wherein the first natural language processing model is pre-trained using a first training set, and the first prior information is determined based on the first training set; The first natural language processing result is sent to the vehicle to complete the voice interaction.
2. The method according to claim 1, characterized in that The samples in the first training set include a plurality of semantic units, and the method further includes: Determining first occurrence frequencies of a plurality of preset text units in the first training set according to samples in the first training set, wherein the preset text units include at least two semantic units; According to a predetermined first frequency upper limit and a first frequency lower limit, smoothing is performed on the first occurrence frequency of each of the preset text units to determine a smoothed frequency; Determining the occurrence probability of the preset text unit according to the smoothed frequency of the preset text unit and the smoothed frequency of the target text unit corresponding to the preset text unit, wherein a semantic unit in the preset text unit is the same as a semantic unit in the target text unit; The first priori information is determined according to the occurrence probability of each of the preset text units.
3. The method according to claim 1, characterized in that: The determining, according to the voice request, the first prior information and the first natural language processing model, a first natural language processing result corresponding to the voice request includes: Predicting a semantic unit corresponding to the voice request according to the voice request and the first natural language processing model, and determining first prediction probability information; Determining second prediction probability information according to the first prediction probability information and the first priori information; The first natural language processing result is determined according to the second prediction probability information.
4. A voice interaction method, characterized in that: For a vehicle, the method comprises: Get voice request; When a first natural language processing result sent by the server can be received, the voice interaction is completed according to the first natural language processing result, wherein the first natural language processing result is obtained by the server processing the voice request according to the voice interaction method according to any one of claims 1-3.
5. The method according to claim 4, characterized in that Also includes: Determine a second natural language processing result corresponding to the voice request according to the voice request, the second prior information and the second natural language processing model, wherein the second natural language processing model is pre-trained using a second training set, and the second prior information is determined based on the second training set; In the event that the first natural language processing result cannot be received, the voice interaction is completed according to the second natural language processing result.
6. The method according to claim 5, characterized in that The samples in the second training set include a plurality of semantic units, and the method further includes: Determining a second occurrence frequency of each semantic unit in the second training set according to samples in the second training set; According to a predetermined second frequency upper limit and a second frequency lower limit, smoothing the second occurrence frequency of each semantic unit in the second training set to determine a processed frequency of each semantic unit in the second training set; Determining the occurrence probability of each semantic unit according to the processed frequency of each semantic unit in the second training set; The second prior information is determined according to the occurrence probability of each semantic unit in the second training set.
7. The method according to claim 5, characterized in that The determining, according to the voice request, the second prior information and the second natural language processing model, a second natural language processing result corresponding to the voice request includes: Predicting a semantic unit corresponding to the voice request according to the voice request and the second natural language processing model, and determining third prediction probability information; Determining fourth prediction probability information according to the third prediction probability information and the second priori information; The second natural language processing result is determined according to the fourth prediction probability information.
8. The method according to claim 5, characterized in that The method further comprises: The second prior information is updated according to the acquired historical voice requests.
9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 8 is implemented.
