Vehicle voice optimization method, device, electronic device and storage medium

Abstract

The vehicle voice optimization method, device, electronic device and storage medium provided by the present invention include: intercepting the voice performance data generated by the vehicle-mounted terminal in response to the user voice at the current moment; determining the processing stage where the voice performance does not meet the standard according to the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user voice; optimizing the voice processing result of the processing stage where the voice performance does not meet the standard according to the user voice, so that the voice performance of the processing stage is up to standard. The present application first intercepts the voice performance data generated by the vehicle-mounted terminal in response to the user voice at the current moment, and then determines the stage that needs to be optimized and the time point when the problem occurs based on the voice performance data, and then optimizes in combination with the user voice. Compared with the existing technology, on the one hand, it solves the current lack of voice service monitoring and analysis capabilities, and the problem of being unable to troubleshoot voice errors.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition, and in particular to a vehicle speech optimization method, device, electronic equipment and storage medium. Background Art

[0002] Traditional cars are developing in the direction of intelligence, and currently most in-vehicle voice recognition needs to be bound to fixed third-party software suppliers. Its voice processing relies entirely on third-party software suppliers. On the one hand, the voice recognition performance of third-party software suppliers varies, and on the other hand, it cannot be fully matched with its own vehicle and cannot be set up specifically for its own vehicle. In addition, since there is currently no better voice error troubleshooting solution, third-party software suppliers are currently unable to troubleshoot voice errors, so there are many deficiencies. Summary of the invention

[0003] In response to the problems in the prior art, the present invention provides a vehicle voice optimization method, device, electronic device and storage medium, aiming to solve the current problems that, on the one hand, the voice recognition performance of third-party software suppliers varies, and on the other hand, it cannot be fully matched with the vehicle itself and cannot be specifically set for the vehicle itself. In addition, since there is currently no better voice error troubleshooting solution, third-party software suppliers are currently unable to troubleshoot at least one of the voice error problems.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] The first aspect of the present application provides a vehicle voice optimization method, which is performed by a vehicle voice optimization device, and includes:

[0006] Intercepting the voice performance data generated by the vehicle-mounted terminal in response to the user's voice at the current moment; the voice performance data is the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user's voice;

[0007] Determine, based on the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user voice, a processing stage in which the voice performance does not meet the standard;

[0008] The speech processing result of the processing stage where the speech performance does not meet the standard is optimized according to the user speech, so that the speech performance of the processing stage is up to the standard.

[0009] In an optional embodiment, each processing stage corresponds to a preset duration, and determining the processing stage where the voice performance does not meet the standard according to the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user voice includes:

[0010] For each processing stage, if the duration corresponding to the processing stage is greater than the preset duration corresponding to the processing stage, it is determined that the speech performance of the processing stage does not meet the standard.

[0011] In an optional embodiment, the processing stage includes a speech recognition stage;

[0012] The speech processing result of the processing stage of optimizing the speech performance not meeting the standard according to the user speech includes:

[0013] Convert the user's speech into a question sentence, and segment the question sentence to obtain multiple segmented words;

[0014] Perform feature processing on each word to obtain the feature vector of each word;

[0015] All feature vectors are input into a preset semantic recognition model, and the semantic recognition model outputs the semantic information of the question sentence to obtain the speech processing result.

[0016] In an optional embodiment, after outputting the semantic information of the question sentence, the vehicle speech optimization method further includes:

[0017] Determining user needs according to semantic information corresponding to the user voice;

[0018] According to the user demand, searching for a third-party information source corresponding to the user demand from a corresponding relationship table between the user demand and the third-party information source;

[0019] The service of the third-party information source is called to respond to the user voice.

[0020] In an alternative embodiment,

[0021] The speech processing result of the processing stage of optimizing the speech performance not meeting the standard according to the user speech includes:

[0022] Selecting an information service channel according to the user voice; each information service channel corresponds to a public cloud server, and the public cloud server is configured with a voice search service;

[0023] A service call instruction is sent to the public cloud server to call the selected voice search service, generate a call result in combination with the user voice, and use the call result as the voice processing result.

[0024] In an alternative embodiment,

[0025] The speech processing result of the processing stage of optimizing the speech performance not meeting the standard according to the user speech includes:

[0026] The user voice is input into each public cloud server through each information service channel, and each public cloud server generates a candidate voice processing result based on the user voice; each information service channel corresponds to a public cloud server;

[0027] One is selected from a plurality of candidate speech processing results based on semantic keywords of the user's speech to obtain the speech processing result.

[0028] A second aspect of the present application provides a vehicle voice optimization device, including:

[0029] An interception module, which intercepts the voice performance data generated by the vehicle terminal in response to the user's voice at the current moment; the voice performance data is the duration of each processing stage in the process of the vehicle terminal responding to the user's voice;

[0030] A determination module, which determines the processing stage where the voice performance does not meet the standard according to the duration of each processing stage in the process of the vehicle terminal responding to the user voice;

[0031] The optimization module optimizes the speech processing result of the processing stage where the speech performance does not meet the standard according to the user speech, so that the speech performance of the processing stage is in a state of meeting the standard.

[0032] A third aspect of the present application provides a vehicle voice interaction system, comprising: a vehicle terminal 1, a vehicle voice optimization device 2, and a plurality of cloud servers 3;

[0033] The vehicle terminal is used to pick up the user's voice, and after recognizing the user's voice, sends a corresponding service call instruction to at least one of the multiple cloud servers according to the user's voice; the multiple cloud servers receive the service call instruction of the vehicle terminal, call the corresponding service, and send the service call result to the vehicle terminal, so that the vehicle terminal responds to the user's voice;

[0034] The vehicle voice optimization device is used to intercept the voice performance data generated by the vehicle terminal in response to the user voice at the current moment; and determine the processing stage where the voice performance does not meet the standard based on the duration of each processing stage in the process of the vehicle terminal responding to the user voice; then optimize the voice processing results of the processing stage where the voice performance does not meet the standard based on the user voice; the voice performance data is the duration of each processing stage in the process of the vehicle terminal responding to the user voice.

[0035] The user voice is input into each public cloud server through each information service channel. In another aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the vehicle voice optimization method is implemented when the processor executes the program.

[0036] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, a vehicle speech optimization method is implemented.

[0037] It can be seen from the above technical scheme that the vehicle voice optimization method, device, electronic device and storage medium provided by the present invention first intercept the voice performance data generated by the vehicle-mounted terminal in response to the user voice at the current moment, then determine the stage that needs to be optimized based on the voice performance data, and then optimize it in combination with the user voice. Compared with the existing technology, on the one hand, it solves the current lack of voice service monitoring and analysis capabilities, and the problem of inability to detect voice errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 The figure is a flow chart of a vehicle speech optimization method in an embodiment of the present invention.

[0040] Figure 2 In the embodiment of the present invention Figure 1 Specific flow chart of step S3 in FIG.

[0041] Figure 3a Schematic diagram of an application scenario in an embodiment of the present invention.

[0042] Figure 3b Schematic diagram of the network architecture in an embodiment of the present invention.

[0043] Figure 3c Schematic diagram of the non-standard processing stage and the standard processing stage in an embodiment of the present invention.

[0044] Figure 4 It is a structural schematic diagram of a vehicle voice optimization device in an embodiment of this aspect.

[0045] Figure 5 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Currently, most in-vehicle voice recognition needs to be bound to a fixed third-party software supplier, and its voice processing is completely dependent on the third-party software supplier. On the one hand, the voice recognition performance of third-party software suppliers varies, and on the other hand, it cannot be fully matched with its own vehicle and cannot be set up specifically for its own vehicle. In addition, since there is currently no better voice error troubleshooting solution, third-party software suppliers are currently unable to troubleshoot voice errors.

[0048] The core concept of this application is to capture the voice performance data generated by the on-board terminal in response to the user's voice at the current moment in real time or at each time interval, and then find the duration of each processing stage from the voice performance data. When the duration is higher than the set value, it can be determined that there is an obstacle to voice recognition here, and the user's voice can be further combined to "regenerate" the response voice, that is, to regenerate a more optimized answer to save it for direct use when a similar voice appears next time.

[0049] Figure 3a The application scenario of the embodiment of the present application is shown, such as Figure 3a As shown, the interactive system of the present application specifically includes: a vehicle-mounted terminal 1, a vehicle voice optimization device 2 and multiple cloud servers 3.

[0050] The vehicle-mounted terminal is specifically used to pick up the user's voice, and after recognizing the user's voice, send a corresponding service call instruction to at least one of the multiple cloud servers according to the user's voice.

[0051] The plurality of cloud servers receive the service calling instruction of the vehicle terminal, call the corresponding service, and send the service calling result to the vehicle terminal, so that the vehicle terminal responds to the user voice;

[0052] The vehicle voice optimization device is used to intercept the voice performance data generated by the vehicle terminal in response to the user voice at the current moment; and determine the processing stage where the voice performance does not meet the standard based on the duration of each processing stage in the process of the vehicle terminal responding to the user voice; then optimize the voice processing results of the processing stage where the voice performance does not meet the standard based on the user voice; the voice performance data is the duration of each processing stage in the process of the vehicle terminal responding to the user voice.

[0053] Specifically, the vehicle voice optimization device 2 in one embodiment is equivalent to a transfer device, combined with Figure 1 As shown, the vehicle voice optimization device 2 transmits the cloud service called by each public cloud server to the vehicle terminal, and further, combined with Figure 3b It can be seen that the vehicle voice optimization device of the present application can further include functions such as device authentication and load balancing. Please refer to the detailed description of subsequent embodiments.

[0054] The following is a detailed description of the method performed by the vehicle voice optimization device of the present application, which is performed by the vehicle voice optimization device. The vehicle voice optimization device can be a cloud server, or a processor or controller on the vehicle side.

[0055] like Figure 1 As shown, an embodiment of one aspect of the present invention provides a vehicle speech optimization method, comprising:

[0056] S1: intercepting the voice performance data generated by the vehicle-mounted terminal in response to the user's voice at the current moment; the voice performance data is the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user's voice.

[0057] S2: determining a processing stage where voice performance does not meet the standard according to the duration of each processing stage in the process of the vehicle terminal responding to the user voice;

[0058] S3: Optimizing the speech processing result of the processing stage where the speech performance does not meet the standard according to the user speech, so that the speech performance of the processing stage is up to standard.

[0059] The vehicle voice optimization method provided by the present invention first intercepts the voice performance data generated by the vehicle-mounted terminal in response to the user voice at the current moment, then determines the stage that needs to be optimized and the time point when the problem occurs based on the voice performance data, and then optimizes in combination with the user voice. Compared with the existing technology, it solves the current lack of voice service monitoring and analysis capabilities and the inability to troubleshoot voice errors.

[0060] In the embodiments of the present application, the vehicle-mounted terminal may be a vehicle computer (i.e., a vehicle operating system) installed on a vehicle carrier, or a mobile terminal connected to a car, such as a mobile phone or a portable computer, and the present application does not impose any restrictions on this.

[0061] The mobile terminal and the car can be connected via short-range wireless communication, Bluetooth, or wired connection, and this application does not impose any restrictions on this.

[0062] Exemplarily, the mobile terminal is connected to the car via Bluetooth, and the mobile terminal can further unlock and lock the car doors and pre-cool and pre-heat the vehicle air conditioner.

[0063] In the embodiment of the present application, user voice refers to the command voice issued by the driver or passenger, such as "turn on the air conditioner", "how is the weather today", etc.

[0064] The user voice of the present application generally includes an identification name, such as "xxx, turn on the air conditioner", where xxx is the identification name. This can accurately avoid the function from being accidentally turned on during communication between users (such as communication between drivers and passengers) without a corresponding prediction program.

[0065] Furthermore, based on a maturely trained system, the present application can also be implemented in a predictive manner instead of using name recognition, for example, recognizing the user's semantics, and then determining whether it is a chat between users or a user instruction based on the semantics.

[0066] Exemplarily, the number of drivers and passengers in the cabin can be further used as an influencing parameter to determine whether it is a chat between users or a user instruction. For example, when there is only one driver, excluding phone calls and singing, it can be concluded that the driver's voice is the user's voice.

[0067] In this embodiment, the analysis can be performed in combination with the weight of each influencing parameter, that is, the duration of the voice, the number of users in the cabin, the current telephone communication situation, and the time interval between the user's previous and next sentences can each be assigned a weight or empirical coefficient, and then whether to issue a command voice can be determined based on the calculation results.

[0068] It can be understood that in the voice recognition process, when the vehicle-mounted terminal responds or responds through a third-party software supplier, it generally includes a voice recognition stage and a result output stage. The voice recognition stage is to recognize the voice content, and the result output stage is to give the response content based on the recognized voice content.

[0069] The inventor of the present application innovatively intercepts voice performance data to obtain the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user's voice, and then determines whether a fault occurs in the processing stage based on the duration.

[0070] It can be understood that the speech processing result is the recognition result or response result given based on the user's voice. In the speech recognition stage, the speech processing result of that stage, that is, the semantic content, is given, and in the result output stage, the speech response result is given.

[0071] The optimization in the present application may be to modify the corresponding program code, or to pre-store the response result and form a corresponding relationship that matches the user's voice, and then call the response result the next time the user makes a similar voice.

[0072] In an optional embodiment, each processing stage corresponds to a preset duration, and determining the processing stage where the voice performance does not meet the standard according to the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user voice includes:

[0073] For each processing stage, if the corresponding duration is longer than the corresponding preset duration, it is determined that the speech performance of the processing stage does not meet the standard.

[0074] For example, if the preset duration of a processing stage is 700 ms, if the duration of the current processing stage is captured to be longer than 700 ms, it is determined that there is an obstacle in the processing stage.

[0075] like Figure 3c As shown, Figure 3c The processing time of each processing stage according to the analysis of the voice performance data in the embodiment of the present application is shown. The circled part in the figure means that the processing time exceeds the set time, that is, the stage is judged as a non-compliant stage.

[0076] Combine the following Figure 3c The specific processing process of this application is described in detail, such as Figure 3c As shown, the vehicle-mounted optimization device of the present application first captures the voice performance data generated by the vehicle-mounted terminal (for example, it can be a car computer) in response to the user's voice at the current time (for example, December 12, 12:10:02), and the voice performance data can be the processing time.

[0077] Exemplarily, the processing stage may include converting speech into text sentences, breaking down the words in the text sentences for semantic analysis, organizing response sentences based on the semantic analysis, and converting the response sentences into speech.

[0078] In certain embodiments, the speech performance data of converting the intercepted speech into text sentences, splitting the text sentences into words for semantic analysis, organizing the response sentences based on the semantic analysis, and converting the response sentences into speech are: 500ms, 1000ms, 1200ms and 300ms respectively.

[0079] Exemplarily, the preset durations for converting speech into text sentences, splitting the words of text sentences for semantic analysis, organizing response sentences based on semantic analysis, and converting response sentences into speech are 700ms, 700ms, 1000ms, and 500ms, respectively. It can be seen at this time that if the processing duration of the stages of splitting the words of text sentences for semantic analysis and organizing response sentences based on semantic analysis is longer than the preset duration, it is defined as not meeting the standard. If the processing duration of the two stages of converting speech into text sentences and converting response sentences into speech is less than the preset duration, it is defined as meeting the standard.

[0080] At this time, after interception, the stage of semantic analysis of word segmentation of split text sentences and the stage of organizing response statements based on semantic analysis can be optimized. For example, the algorithm of semantic analysis and the database of response statement correspondence can be replaced.

[0081] It can be understood that in the embodiments of the present application, the speech processing result is the output of the speech processing data at each stage. For example, in the processing stage of converting speech into text sentences, the speech processing result is a text sentence, and the speech processing result of the processing stage of splitting the text sentences into words for semantic analysis is the semantic content. This application does not list them all.

[0082] Exemplarily, the above-mentioned processing stages of the present application are defined according to the processing process of the data content. In other embodiments, they can also be defined based on the processing status of the voice data. For example, the processing stages of the voice data include: voice recognition on the screen, voice recognition completion, and dialogue result output stages.

[0083] It can be understood that the duration of the above-mentioned processing stages can be intercepted from the vehicle-mounted terminal. The vehicle-mounted terminal will generate a processing log or give an identification of the processing stage during processing. That is, after completing a processing stage, the vehicle-mounted terminal generates an identification, instruction or processing log. The vehicle optimization device can determine the duration of the processing stage by intercepting the identification, instruction or processing log.

[0084] In an optional embodiment, the processing stage includes a speech recognition stage, and the speech processing result of the speech recognition stage is semantic information;

[0085] The speech processing result of the processing stage of optimizing the speech performance not meeting the standard according to the user speech includes:

[0086] S31: converting the user's speech into a question sentence, and segmenting the question sentence to obtain a plurality of segmented words;

[0087] S32: Perform feature processing on each segmented word to obtain a feature vector for each segmented word;

[0088] S33: All feature vectors are input into a preset semantic recognition model, and the semantic recognition model outputs semantic information of the question sentence.

[0089] In this embodiment, a built-in semantic recognition model is used and then the model is called to perform semantic recognition. On the one hand, inaccurate semantic recognition by third-party software vendors can be avoided. On the other hand, optimization can be performed using its own data and user feedback, thereby making semantic recognition more and more accurate.

[0090] Further, in an optional embodiment, the semantic recognition model is a neural network model, and the method further includes:

[0091] Establishing the neural network model;

[0092] The neural network model is trained using word groups with labeled semantic information, wherein the neural network model after training convergence is the semantic recognition model.

[0093] In this embodiment, the semantic recognition model can be a neural network model, specifically an artificial neural network model (ANN), a convolutional neural network model (CNN) or a probabilistic neural model (PNN), and this application does not impose any restrictions on this.

[0094] For example, specifically, the semantic recognition model is a CNN model. That is, when training is first performed, each word combination is marked with a response sentence, and then the response sentence is identified, such as 1, 2, 3, 4, that is, the word combination is corresponded to 1, 2, 3, 4, etc., and then they are all input into the CNN model, and then through convolution and pooling, the weight of the entire network is obtained. After the training is stable, the network weight is stable, so that after inputting different word combinations, a specific response sentence can be given.

[0095] In an optional embodiment, the method further includes: performing a device authority authentication operation on the vehicle-mounted terminal, and if the authentication is successful, sending the voice processing result to the vehicle-mounted terminal.

[0096] In this embodiment, the device is authenticated to prevent non-vehicle terminal devices from accessing.

[0097] Furthermore, the processing stage of the present application includes a result output stage, as shown in Table 1 below, the voice recognition on screen and the voice recognition completion belong to the voice recognition stage, totaling 700ms, and the output of the dialogue result is the result output stage.

[0098] Table 1 - Processing Phase Timeline

[0099] Speech recognition screen asr_final_elapsed Standard 200ms Speech recognition completed first_asr_text_elapsed Standard 500ms Dialogue result output dm_final_elapsed Standard 700ms

[0100] Furthermore, in the embodiments of the present application, various public clouds on the market can be connected and the services of the public clouds can be called. For example, the services of public cloud A include: authentication services, automatic speech recognition (ASR) services, ASR training, skill central control, DM service central control, skill semantic services, skill platforms and information source services, etc. This application does not elaborate on this.

[0101] Furthermore, the present application can access the B public cloud, that is, the present application can access multiple public clouds at the same time, that is, the processing stage includes the result output stage;

[0102] The vehicle voice optimization device includes a network interface for accessing multiple information service channels;

[0103] The speech processing result of the processing stage of optimizing the speech performance not meeting the standard according to the user speech includes:

[0104] Selecting an information service channel according to the user voice; each information service channel corresponds to a public cloud server and includes a voice search service;

[0105] A service call instruction is sent to the public cloud server to call the selected voice search service, generate a call result in combination with the user voice, and use the call result as the voice processing result.

[0106] This embodiment can select a public cloud server with stronger voice recognition, use the voice search service of the public cloud server, and process and recognize the voice through the higher processing performance of the public cloud server, thereby further improving the accuracy and speed of voice recognition.

[0107] Furthermore, in some embodiments, after obtaining the semantic information, the vehicle speech optimization method further includes:

[0108] Determining user needs according to semantic information corresponding to the user voice;

[0109] According to the user's needs, a third-party information source corresponding to the user's needs is searched from a corresponding relationship table between the user's needs and the third-party information source;

[0110] The service of the third-party information source is called to respond to the user voice.

[0111] This embodiment is based on a correspondence table between the semantic information of the user's voice and the third-party information source. In this embodiment, an information service channel is selected according to the user's voice, that is, the corresponding public cloud server is selected, and the corresponding service is searched from the public cloud server for calling, so that a variety of different public cloud servers can be matched. The best public cloud server can be called for each type of user demand. For example, public cloud A has the best weather forecast service on the market. When a user needs to inquire about the weather, the weather forecast service on public cloud A can be called.

[0112] In this embodiment, multiple information service channels can be connected, such as the public cloud mentioned above, and then the terminal device selects an information service channel. At this time, the specific selection process includes: analyzing the semantics of the user's voice, then matching the user's needs, and then selecting the information service channel based on the needs, such as weather forecast services, road warning services, etc.

[0113] In an optional embodiment, the processing stage includes a result output stage. For example, the preset duration of the result output stage in Table 1 above is 700ms. The vehicle voice optimization device includes a network interface for access to multiple information service channels. The voice processing results of the processing stage in which the voice performance does not meet the standard is optimized according to the user voice include: inputting the user voice into each public cloud server through each information service channel, and each information service channel generates a voice processing result based on the user voice; and selecting one from multiple voice processing results based on the semantic keywords of the user voice.

[0114] This embodiment sends the voice to all information service channels, and then each information service channel generates a voice processing result based on the user voice, and then selects one from multiple voice processing results based on the semantic keywords of the user voice, that is, screening is performed by whether the feedback voice processing result matches the user voice.

[0115] Combine the following Figure 3a and Figure 3b The interactive process of this application is described in detail. Figure 3b The CRRC terminal is the vehicle-mounted terminal of this application. The entire interaction process is as follows:

[0116] Step 1: The driver or passenger issues a voice command, which can be set to include certain response words, for example: Xiao XX, how is the weather tomorrow? Xiao XX is the response word, which can be recognized by the vehicle terminal, and the following voice is defined as the voice that the user needs to interact with the vehicle terminal.

[0117] Step 2: The vehicle terminal receives the message “Xiao XX, how is the weather tomorrow?” and determines through the response word “Xiao XX” that “How is the weather tomorrow?” in the user's voice is the instruction that the user wants to interact with.

[0118] Step 3: The vehicle terminal displays "What will the weather be like tomorrow?" on the display screen to ensure that the user can adjust or re-send the voice in case of recognition errors. At this time, the system generates a timestamp for the screen display, which can be generated through logs.

[0119] Step 4: The voice optimization device obtains the log of the vehicle terminal in real time. Before the event is over, the log has not been generated and cannot be obtained. After the vehicle terminal generates the timestamp of the screen printing through the log, the timestamp is obtained immediately, and the processing time of the screen printing stage can be determined. Then it is compared with the preset processing time to determine whether it meets the standard. If it meets the standard, proceed to step 4. If it does not meet the standard, it is necessary to optimize the algorithm of the vehicle terminal during the screen printing process, such as replacing it with a faster and more targeted algorithm.

[0120] Step 4: After the vehicle terminal confirms that "What's the weather like tomorrow?" is the user's voice (that is, the user has not modified it within the specified time), the internal semantic parsing program starts semantic parsing, confirms that the user wants to know tomorrow's weather, and generates a processing log, which includes the process timestamp of the semantic parsing.

[0121] Step 5: The voice optimization device obtains the log of the vehicle terminal in real time. Before the event is over, the log has not been generated and cannot be obtained. After the vehicle terminal obtains the log, the process timestamp of the semantic analysis is found. If it meets the standard, proceed to step 6. If it does not meet the standard, it is necessary to optimize the algorithm of the vehicle terminal during the semantic analysis process, such as replacing it with a faster and more targeted algorithm.

[0122] Step 6: After each processing stage, the vehicle terminal outputs a response statement and transmits it to the vehicle optimization device.

[0123] Step 7: When the vehicle optimization device outputs a response statement on the vehicle terminal, it can call the services of various public clouds based on the semantic content.

[0124] The following is a detailed description of the vehicle optimization device calling service.

[0125] 1. When the vehicle optimization device accesses the public cloud server, it can first authenticate with the public cloud server to confirm whether the public cloud server has the permission to access. Further, when the vehicle optimization device accesses the vehicle terminal, it can authenticate with the vehicle terminal.

[0126] 2. During the specific processing process, the vehicle optimization device further performs load balancing control and flow control on the entire processing process.

[0127] 3. The vehicle optimization device can provide transfer services, connecting the public cloud services to the vehicle terminal through the information service channel, and can realize routing terminal and data transfer.

[0128] 4. The public cloud can include a variety of services. The public cloud server can first include a speech recognition (Automatic Speech Recognition, ASR) service. The speech recognition service can be called by the vehicle terminal through the vehicle optimization device and called when the speech recognition is performed. Or when the vehicle optimization device detects that the speech recognition of the vehicle terminal does not meet the standards, the speech recognition service of the public cloud server can be used to replace the speech recognition algorithm of the vehicle terminal, thereby calling the speech recognition result from the public cloud server and transmitting it to the vehicle terminal.

[0129] 5. The public cloud server further includes ASR training services, which can perform model training for semantic recognition, skill central control services, and skill feedback control after semantic recognition. For example, semantic recognition of the weather can provide weather forecast skills, user registration service central control, skill semantic services, skill platform, and information source services, which are not elaborated in this application.

[0130] It can be seen from the above embodiments that the vehicle voice optimization method provided by the present invention first intercepts the voice performance data generated by the vehicle terminal in response to the user voice at the current moment, and then determines the stage that needs to be optimized and the time point when the problem occurs based on the voice performance data, and then optimizes in combination with the user voice. Compared with the prior art, on the one hand, it solves the current lack of voice service monitoring and analysis capabilities, and the problem of being unable to troubleshoot voice errors. The present application provides a vehicle voice optimization device at the software level, including:

[0131] An interception module 1 intercepts the voice performance data generated by the vehicle-mounted terminal in response to the user's voice at the current moment; the voice performance data is the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user's voice;

[0132] Determining module 2, determining a processing stage where voice performance does not meet the standard according to the duration of each processing stage in the process of the vehicle terminal responding to the user voice;

[0133] The optimization module 3 optimizes the speech processing result of the processing stage where the speech performance does not meet the standard according to the user speech, so that the speech performance of the processing stage is in a state of meeting the standard.

[0134] It can be understood that the device can be integrated inside the vehicle-mounted terminal or can be communicatively connected to the vehicle-mounted terminal, and this application does not impose any restrictions on this.

[0135] It can be seen from the above embodiments that the vehicle voice optimization device provided by the present invention, by configuring an acquisition module, a negative emotion determination module and a vehicle voice optimization module, first intercepts the voice performance data generated by the vehicle-mounted terminal in response to the user voice at the current moment, and then determines the stage that needs to be optimized and the time point when the problem occurs based on the voice performance data, and then optimizes in combination with the user voice. Compared with the existing technology, on the one hand, it solves the current lack of voice service monitoring and analysis capabilities, and the problem of being unable to detect voice errors.

[0136] From the hardware level, the present invention provides an embodiment of an electronic device for implementing all or part of the vehicle voice optimization method, and the electronic device specifically includes the following contents:

[0137] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between servers, devices, distributed message middleware cluster devices, various databases and user terminals and other related devices; the electronic device can be a desktop computer, a tablet computer and a mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the electronic device can be implemented with reference to the embodiment of the vehicle voice optimization method and the embodiment of the vehicle voice optimization device in the embodiment, and the contents thereof are incorporated herein and the repeated parts are not repeated.

[0138] Figure 5 FIG. 9 is a schematic block diagram of a system structure of an electronic device 9600 according to an embodiment of the present invention. Figure 5 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Figure 5 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0139] In one embodiment, the vehicle voice optimization function can be integrated into the central processor 9100.

[0140] In another embodiment, the vehicle voice optimization device can be configured separately from the central processing unit 9100. For example, the vehicle voice optimization device can be configured as a chip connected to the central processing unit 9100, and the vehicle voice optimization function can be implemented through the control of the central processing unit.

[0141] like Figure 5 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Figure 5 In addition, the electronic device 9600 may also include Figure 5 For components not shown, reference may be made to the prior art.

[0142] like Figure 5 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0143] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0144] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0145] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0146] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0147] The communication module 9110 is a transmitter / receiver 9110 that sends and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0148] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module, etc. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0149] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all the steps of the vehicle voice optimization method in the above embodiment, wherein the execution subject may be a server, and a computer program is stored on the computer-readable storage medium, which implements all the steps of the vehicle voice optimization method in the above embodiment when executed by a processor.

[0150] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A vehicle voice optimization method, characterized in that: Applicable to vehicle voice optimization devices, including: Intercepting the voice performance data generated by the vehicle-mounted terminal in response to the user's voice at the current moment; the voice performance data is the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user's voice; Determine, based on the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user voice, a processing stage in which the voice performance does not meet the standard; Optimizing the speech processing result of the processing stage where the speech performance does not meet the standard according to the user speech, so that the speech performance of the processing stage is in a state of meeting the standard; The speech processing result of the processing stage of optimizing the speech performance not meeting the standard according to the user speech includes: The user voice is input into each public cloud server through each information service channel, and each public cloud server generates a candidate voice processing result based on the user voice; each information service channel corresponds to a public cloud server; One is selected from a plurality of candidate speech processing results based on semantic keywords of the user's speech to obtain the speech processing result.

2. The vehicle voice optimization method according to claim 1, characterized in that: Each processing stage corresponds to a preset duration. The process of determining the processing stage where the voice performance does not meet the standard according to the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user voice includes: For each processing stage, if the duration corresponding to the processing stage is greater than the preset duration corresponding to the processing stage, it is determined that the speech performance of the processing stage does not meet the standard.

3. The vehicle voice optimization method according to claim 2, characterized in that: The processing stage includes a speech recognition stage; The speech processing result of the processing stage of optimizing the speech performance not meeting the standard according to the user speech includes: Convert the user's speech into a question sentence, and segment the question sentence to obtain multiple segmented words; Perform feature processing on each word to obtain the feature vector of each word; All feature vectors are input into a preset semantic recognition model, and the semantic recognition model outputs the semantic information of the question sentence to obtain the speech processing result.

4. The vehicle voice optimization method according to claim 3, characterized in that: After outputting the semantic information of the question sentence, the vehicle speech optimization method further includes: Determining user needs according to semantic information corresponding to the user voice; According to the user demand, searching for a third-party information source corresponding to the user demand from a corresponding relationship table between the user demand and the third-party information source; The service of the third-party information source is called to respond to the user voice.

5. The vehicle voice optimization method according to claim 1, characterized in that: The speech processing result of the processing stage of optimizing the speech performance not meeting the standard according to the user speech includes: Selecting an information service channel according to the user voice; each information service channel corresponds to a public cloud server, and the public cloud server is configured with a voice search service; A service call instruction is sent to the public cloud server to call the selected voice search service, generate a call result in combination with the user voice, and use the call result as the voice processing result.

6. A vehicle voice optimization device, characterized in that: include: An interception module, which intercepts the voice performance data generated by the vehicle terminal in response to the user's voice at the current moment; The voice performance data is the duration of each processing stage in the process of the vehicle terminal responding to the user voice; A determination module, which determines the processing stage where the voice performance does not meet the standard according to the duration of each processing stage in the process of the vehicle terminal responding to the user voice; An optimization module, which optimizes the speech processing result of the processing stage where the speech performance does not meet the standard according to the user speech, so that the speech performance of the processing stage is in a state of meeting the standard; The speech processing result of the processing stage of optimizing the speech performance not meeting the standard according to the user speech includes: The user voice is input into each public cloud server through each information service channel, and each public cloud server generates a candidate voice processing result based on the user voice; each information service channel corresponds to a public cloud server; One is selected from a plurality of candidate speech processing results based on semantic keywords of the user's speech to obtain the speech processing result.

7. A vehicle voice interaction system, characterized in that: include: In-vehicle terminal, vehicle voice optimization device and multiple cloud servers; The vehicle terminal is used to pick up the user's voice, and after recognizing the user's voice, sends a corresponding service call instruction to at least one of the multiple cloud servers according to the user's voice; the multiple cloud servers receive the service call instruction of the vehicle terminal, call the corresponding service, and send the service call result to the vehicle terminal, so that the vehicle terminal responds to the user's voice; The vehicle voice optimization device is used to intercept the voice performance data generated by the vehicle terminal in response to the user's voice at the current moment; and determining the processing stage where the voice performance does not meet the standard according to the duration of each processing stage in the process of the vehicle-mounted terminal responding to the user voice; Then, optimizing the speech processing results of the processing stage where the speech performance does not meet the standard according to the user speech; The voice performance data is the duration of each processing stage in the process of the vehicle terminal responding to the user voice; The speech processing result of the processing stage in which the speech performance does not meet the standard is optimized according to the user speech, comprising: inputting the user speech into each public cloud server through each information service channel, each public cloud server generating a candidate speech processing result based on the user speech; each information service channel corresponds to a public cloud server; and selecting one from a plurality of candidate speech processing results based on semantic keywords of the user speech to obtain the speech processing result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the vehicle voice optimization method described in any one of claims 1 to 5 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the vehicle speech optimization method according to any one of claims 1 to 5 is implemented.

Images