Voice instruction feedback method, vehicle machine and program product

Abstract

The invention discloses a voice instruction feedback method, a vehicle machine and a program product, and relates to the technical field of intelligent cabins, and the method comprises the steps: inputting a voice instruction into a large model; according to a feedback text of the large model, pre-judging execution of a voice instruction, related background service and background data; and pre-loading the background service and the background data. According to the voice instruction feedback method, the vehicle machine and the program product provided by the invention, the execution efficiency of the voice instruction in the cabin can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent cockpit, in particular to a voice instruction feedback method, an in-vehicle infotainment system and a program product. BACKGROUND

[0002] With the progress of intelligent cockpit technology, voice interaction has gradually become the mainstream way of human-vehicle interaction. However, the existing voice system still has many limitations in response speed and accuracy, especially when dealing with complex instructions, the system often needs to wait for the voice to be completely parsed before processing, resulting in poor user experience. In addition, for ambiguous and incomplete instructions, the existing system is difficult to feedback or give intelligent suggestions in time, thereby affecting the fluency of instruction execution.

[0003] The deficiencies of the prior art are:

[0004] 1. Slow response speed: the system often performs background processing after the voice instruction is parsed, causing feedback delay.

[0005] 2. Lack of intelligent prediction: the existing system cannot predict the instructions that the user may issue, resulting in the need for the user to confirm complex instructions repeatedly, increasing interaction time.

[0006] 3. Insufficient ability to handle ambiguous instructions: for ambiguous instructions issued by the user, the existing system cannot intelligently correct or provide efficient feedback, affecting user experience. SUMMARY

[0007] The purpose of the present application is to provide a voice instruction feedback method, an in-vehicle infotainment system and a program product, which can improve the execution efficiency of voice instructions in the cockpit.

[0008] The present application provides the following solutions:

[0009] According to one aspect of the present application, a voice instruction feedback method is provided, which comprises:

[0010] inputting a voice instruction into a large model;

[0011] pre-judging the execution of the voice instruction, the background services and the background data involved according to the feedback text of the large model;

[0012] pre-loading the background services and the background data.

[0013] Optionally, the pre-judging the execution of the voice instruction, the background services and the background data involved according to the feedback text of the large model comprises:

[0014] obtaining background service keywords in the feedback text;

[0015] counting the word frequency of the background service keywords;

[0016] Obtaining the background service keywords in the top N positions in the word frequency statistics;

[0017] The background services and background data associated with the top N background service keywords are determined as the background services and background data associated with the execution of the voice instruction.

[0018] Optionally, the method further comprises:

[0019] Intelligently correcting the received incomplete, ambiguous or ambiguous voice instruction.

[0020] Optionally, intelligently correcting the received incomplete, ambiguous or ambiguous voice instruction comprises:

[0021] Obtaining the context of the voice instruction;

[0022] Correcting the voice instruction according to the analysis of the context.

[0023] Optionally, intelligently correcting the received incomplete, ambiguous or ambiguous voice instruction comprises:

[0024] Obtaining a voice instruction similar to the current voice instruction;

[0025] Correcting the current voice instruction using the similar voice instruction.

[0026] Optionally, the method further comprises:

[0027] Providing preliminary feedback on the executed voice instruction.

[0028] Optionally, providing preliminary feedback on the executed voice instruction comprises:

[0029] Providing preliminary feedback on the executed voice instruction according to the current system state.

[0030] According to the two aspects of the present application, a voice instruction feedback device is provided, which comprises:

[0031] An input module for inputting a voice instruction to a large model;

[0032] A pre-judgment module for pre-judging the background services and background data involved in the execution of the voice instruction according to the feedback text of the large model;

[0033] A pre-loading module for pre-loading the background services and background data.

[0034] According to the three aspects of the present application, a car machine is provided, comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the voice instruction feedback method as described above.

[0035] According to the four aspects of the present application, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the voice instruction feedback method as described above.

[0036] Through the above scheme, the following beneficial technical effects are obtained:

[0037] 1. Improve response speed: through the dynamic prediction of instructions, the system can process related tasks in advance when the user's voice input is not completed, greatly improving the response speed.

[0038] 2. Improve instruction execution accuracy: through the intelligent correction function of the large model, the system can effectively process fuzzy and incomplete voice instructions, reduce the user's repeated input demand, and improve the interaction efficiency.

[0039] 3. Enhance user experience: the real-time preliminary feedback mechanism can let the user know the status of the system in real time, reduce the waiting anxiety, and improve the overall interaction experience of the user. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a flowchart of the voice instruction feedback method provided by one or more embodiments of the present application;

[0041] Figure 2 is a flowchart of the prediction operation in the voice instruction feedback method provided by one or more embodiments of the present application;

[0042] Figure 3 is a flowchart of the voice instruction feedback method provided by one or more embodiments of the present application;

[0043] Figure 4 is a flowchart of the intelligent correction operation in the voice instruction feedback method provided by one or more embodiments of the present application;

[0044] Figure 5 is a flowchart of the intelligent correction operation in the voice instruction feedback method provided by one or more embodiments of the present application;

[0045] Figure 6 is a flowchart of the voice instruction feedback method provided by one or more embodiments of the present application;

[0046] Figure 7 is a structural diagram of the voice instruction feedback device provided by one or more embodiments of the present application;

[0047] Figure 8 is a structural diagram of a car machine provided by one or more embodiments of the present application. DETAILED DESCRIPTION

[0048] The technical solutions of the present application will be described below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0049] Figure 1 is a flowchart of a voice instruction feedback method provided by one or more embodiments of the present application. Referring to Figure 1 , the voice instruction feedback method comprises the following operation steps:

[0050] S11, input the voice instruction to the large model.

[0051] S12, according to the feedback text of the large model, pre-judge the execution of the voice instruction, the background service and the background data involved.

[0052] S13, preloading the background service and the background data.

[0053] In the era of Internet of Things and large-scale application of intelligent cockpit, more and more people start to use voice instructions to complete human-computer interaction with vehicles in the cockpit of vehicles.

[0054] However, the use of voice instructions still faces many obstacles. Compared with the input operation directly on the screen of the car machine, the acceptance of voice instructions is still not high. In addition, since the execution of voice instructions is based on the complete analysis of voice by machine, the user's psychological feeling is that the execution speed of voice instructions is not high.

[0055] The technical solution provided by the present embodiment is to reverse such misconception and improve the execution efficiency of voice instructions on the car machine.

[0056] It should be understood that the execution process of voice instructions often involves background services or data. The loading of these background services and background data usually needs to consume system execution time.

[0057] In the present embodiment, first, the large model is used to pre-judge the background services and background data involved in the execution of voice instructions, and to determine which background services or background data will be involved in the execution process of voice instructions.

[0058] After the background data and background services involved in the execution process of voice instructions are determined, the background data and background services are preloaded.

[0059] In this way, when waiting for a voice instruction, the background data and the background service have already been resident in the system memory when the background data and the background service are actually called. Therefore, the technical solution provided by the embodiment can greatly improve the execution speed of the voice instruction and improve the execution efficiency.

[0060] In the entire process described above, the large language model plays an irreplaceable role. The large language model can output a large amount of feedback text on the premise of a small amount of text input. The embodiment takes advantage of the large language model to output text and uses the output text of the large language model as a basis for determining the next background data and background service.

[0061] For example, inputting a voice instruction of "open the sunroof" into the large language model, the large language model will feedback the following text:

[0062] "Open the sunroof" has the following two common understandings:

[0063] Real-world scenarios

[0064] If you say "open the sunroof" to mean opening the sunroof of a house, car, etc. in reality, the following are the operation methods in different scenarios:

[0065] • House sunroof

[0066] • Manual sunroof: Generally equipped with a dedicated crank or handle. When operating, insert the crank into the corresponding socket and turn it in a certain direction (usually clockwise) to gradually open the sunroof; or directly hold the handle and push it upwards or sideways to open the sunroof.

[0067] • Electric sunroof: Usually there is a control switch indoors, usually near the ceiling or wall near the sunroof. Press the open button and the sunroof will automatically open.

[0068] • Car sunroof

[0069] • Most car sunroofs are electric, and the control button is usually located on the front row of the car roof. Press the open button and the sunroof will open according to the preset mode, some can slide backwards, and some can tilt upwards to ventilate.

[0070] Idiomatic level

[0071] "Open the sunroof and speak frankly" is a widely circulated idiom, which means speaking directly and clearly. If you want to express such a direct and frank communication intention in communication, you can say to the other party "let's open the sunroof and speak frankly". For example, "let's open the sunroof and speak frankly, and we must renegotiate the profit distribution of this project".

[0072] It can be seen that the text output by the large language model contains more information than the original voice instruction.

[0073] Based on the text given by the large language model, further analysis can be performed to determine that the execution of the instruction information will involve the sunroof in the cockpit.

[0074] Then, the car machine searches for data, components and services associated with the sunroof in the system.

[0075] The search for associated data, components and services can be performed through the system's historical logs. For example, the system logs can be searched using the keyword "window" to find all log entries containing the keyword. Then, see which system data, system components and system service names are included in these log entries to find the background data and background services associated with the sunroof.

[0076] After finding the corresponding background data and background services, these background data and background services that may be used are preloaded into the system memory, and handles for calling these system resources are provided, thus completing the technical solution provided by the embodiment.

[0077] It should be emphasized that the pre-judgment and pre-loading process of the system resources described above is a parallel process to the recognition and analysis process of the voice instruction, so there will be no phenomenon of slowing down the voice instruction execution process, but only speeding up the voice instruction execution process.

[0078] Figure 2 is a flowchart of the pre-judgment operation in the voice instruction feedback method provided by one or more embodiments of the present application. Referring to Figure 2 , according to the feedback text of the large model, the execution of the voice instruction, the background services and background data involved, include the following operation steps:

[0079] S21, obtaining the background service keywords in the feedback text.

[0080] S22, counting the word frequency of the background service keywords.

[0081] S23, obtaining the background service keywords in the top N positions in the word frequency statistics.

[0082] S24, determining the background services and background data associated with the top N background service keywords as the background services and background data associated with the execution of the voice instruction.

[0083] This embodiment focuses on the execution process of the pre-judgment operation described in the foregoing embodiments of the present application.

[0084] It should be understood that each background service has some keywords associated therewith. These keywords can be used as tags to identify a background service.

[0085] For example, the keywords associated with the background service gearbox include gear shifting, gear shifting, gear set, etc.

[0086] Therefore, if the keywords associated with the gearbox appear frequently in the text fed back by the large language model, it can be determined that the background service associated with the input voice instruction is the gearbox.

[0087] Based on the above principle, after obtaining the output text fed back by the large language, the keywords appearing therein are counted. Specifically, the feedback text can be segmented first, and then the word frequency of the segmentation result is counted.

[0088] The preliminary word frequency count is completely based on the keywords appearing in the feedback text. That is, the word frequency of the keywords appearing in the feedback text is counted.

[0089] After the above preliminary statistics, the original keywords appearing in the feedback text are mapped to different background service keywords. For example, if the keyword gear set appears 4 times in the feedback text, the frequency of the background service keyword gearbox increases by 4 times.

[0090] After the above calculation process, the background service keywords with the highest frequency are the background service and data associated with the input voice instruction.

[0091] In the case where the original keyword may correspond to multiple background service keywords, in the present embodiment, the frequency of the original keyword is mapped to all possible background service keywords, and the final identified background service keyword is determined by the final determination step.

[0092] For example, the original keyword torque may correspond to the background service keyword spindle or the background service keyword steering wheel. Therefore, the frequency of the original keyword torque is mapped to the background service keywords spindle and steering wheel. In the final frequency ranking, it is determined whether the finally identified background service keyword is the spindle or the steering wheel.

[0093] Figure 3 It is a flowchart of the voice instruction feedback method provided by one or more embodiments of the present application. Referring to Figure 3 , the voice instruction feedback method includes the following operation steps:

[0094] S31, input the voice instruction to the large model.

[0095] S32, judging the execution of the voice instruction, the background service and the background data involved according to the feedback text of the large model.

[0096] S33, preloading the background service and the background data.

[0097] S34, intelligently correcting the received incomplete, ambiguous or ambiguous voice instruction.

[0098] In the cockpit, the actual received voice instruction is often an incomplete, ambiguous or ambiguous voice instruction.

[0099] For example, the voice instruction we need to process often appears in the form of "turn off". The action pointed by this voice instruction is definitely to turn off, but who is the object of the turn-off action and whether there are any precautions during the turn-off process are not mentioned. This is a clear example of an incomplete voice instruction.

[0100] There are also some ambiguous and unclear voice instructions. The so-called ambiguity is the ambiguity in voice and the ambiguity in semantics. The ambiguity in voice is that the voice is not clear. The ambiguity in voice is that the semantics is ambiguous. "Save this song", which song is it, a typical example of ambiguous semantics.

[0101] That is, the actual received voice instruction in the cockpit is not directly processed, and some pre-processing is required before entering the subsequent instruction execution process. These voice instructions include incomplete, ambiguous, and ambiguous.

[0102] It should be understood that the intelligent correction of the above problematic voice instructions is completed according to some prior information.

[0103] Figure 4 is the flowchart of the intelligent correction operation in the voice instruction feedback method provided by one or more embodiments of the present application. Referring to Figure 4 Intelligent correction of the received incomplete, ambiguous or ambiguous voice instruction includes the following operation steps:

[0104] S41, obtaining the context of the voice instruction.

[0105] S42, correcting the voice instruction according to the analysis of the context.

[0106] It should be understood that each time the user inputs a voice instruction, there is a certain context.

[0107] The context here can be simply understood as the previous voice command input, the feedback obtained by the previous voice command, the next voice command input, and the feedback obtained by the next voice command input corresponding to the next voice command input.

[0108] For example, before the input of "close the sunroof", the input voice command is "turn on the air conditioner".

[0109] For another example, before the input of the voice command "turn on the cold air of the co-driver", the user has also input the voice command "turn on the cold air of the main driver", and this voice command has not been successfully executed due to a fault in the main driver air outlet.

[0110] It can be understood that in the previous input and next input voice commands and their corresponding execution results, there is a lot of information related to the current voice command input. Through the analysis of this information, the general meaning of the current input voice command can be roughly judged.

[0111] After obtaining the semantics of the current input voice command, accurate correction of the current input voice command can be performed.

[0112] Further, the context information obtained not only includes the previous step and next step voice command input and the corresponding feedback, but also includes the previous N steps and next N steps voice command input and the corresponding feedback.

[0113] Further, the context information obtained can also include the cabin environment parameters of the input voice command. For example, the temperature in the cabin is how much, and the humidity is how much.

[0114] Further, the context information obtained can also include the environment parameters outside the cabin. For example, the weather on the day when the voice command is input is a fine day.

[0115] Figure 5 It is a flowchart of the intelligent correction operation in the voice command feedback method provided by one or more embodiments of the present application. Referring to Figure 5 Intelligent correction is performed on the received incomplete, ambiguous or fuzzy voice command, including the following operation steps:

[0116] S51, obtaining a voice command similar to the current voice command.

[0117] S52, correcting the current voice command using the similar voice command.

[0118] There are many voice commands received in the system. Among the many received voice commands, a voice command similar to the current voice command can always be found.

[0119] The search for the similar voice instruction can be a search based on information within the current voice instruction.

[0120] For example, the first half of the voice of the current voice instruction is unclear, but the second half of the voice is clear. At this time, a voice instruction with the same content as the second half of the current voice instruction can be searched in the voice instructions received by the system in the past.

[0121] For example, the current voice instruction is "what air conditioner". The first half of the voice is unclear, but the two words "air conditioner" are clear. At this time, a voice instruction containing "air conditioner" can be searched in the past voice instructions as a similar voice instruction of the current voice instruction.

[0122] Among the voice instructions found to air conditioners as operation targets, there are three types of actions performed on air conditioners, one is to turn on, one is to turn off, and one is to adjust parameters. Therefore, according to the context of the current voice instruction input, it can be determined what action the current voice instruction needs to perform.

[0123] Figure 6 is a flowchart of the intelligent correction operation in the voice instruction feedback method provided by one or more embodiments of the present application. Referring to Figure 6 , the voice instruction feedback method includes the following operation steps:

[0124] S61, input the voice instruction to the large model.

[0125] S62, according to the feedback text of the large model, pre-judge the execution of the voice instruction, the background service and the background data involved.

[0126] S63, preloading the background service and the background data.

[0127] S64, giving a preliminary feedback to the executed voice instruction.

[0128] The execution of some voice instructions cannot immediately give the user feedback of the execution result. The result of this type of voice instruction needs to be waited for a period of time before the actual execution result can be seen.

[0129] In order to avoid the anxiety caused by long waiting time of the user, the system can give an intermediate feedback information in advance. For example, it can be displayed on the car machine screen: your instruction is being executed, please wait patiently.

[0130] Figure 7 is a structural diagram of the voice instruction feedback device provided by one or more embodiments of the present application. Referring to Figure 7 , the voice instruction feedback device includes:

[0131] Input module 71 is used to input voice commands into the large model.

[0132] The prediction module 72 is used to predict in advance the execution of voice commands, the background services and background data involved, based on the feedback text of the large model.

[0133] The preloading module 73 is used to preload background services and background data.

[0134] It is worth noting that although only some basic functional modules are disclosed in the embodiments of this invention, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, what this embodiment intends to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules should not be considered as the scope of protection of the claims of this invention being limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above device is described separately according to its functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0135] like Figure 8 As shown, the present invention also provides a vehicle infotainment system, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of a voice command feedback method.

[0136] Figure 8 This is a structural schematic diagram of a vehicle-mounted infotainment system provided in an embodiment of the present invention. Figure 8 The structure shown in this embodiment of the invention includes one or more processors 810 and a memory 820; the processors 810 in the vehicle infotainment system can be one or more. Figure 8 Taking a processor 810 as an example; memory 820 is used to store one or more programs; the one or more programs are executed by the one or more processors 810, so that the one or more processors 810 implement the voice command feedback method as described in any one of the embodiments of the present invention.

[0137] The vehicle infotainment system may also include an input device 830 and an output device 840.

[0138] The processor 810, memory 820, input device 830, and output device 840 in the vehicle's infotainment system can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0139] The memory 820 in the vehicle infotainment system serves as a computer-readable storage medium, capable of storing one or more programs. These programs can be software programs, computer-executable programs, or modules, such as the program instructions / modules corresponding to the voice command feedback method provided in this embodiment. The processor 810 executes various functional applications and data processing of the vehicle infotainment system by running the software programs, instructions, and modules stored in the memory 820, thereby implementing the voice command feedback method described in the above embodiment.

[0140] The memory 820 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the vehicle's infotainment system. Furthermore, the memory 820 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 820 may further include memory remotely located relative to the processor 810, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0141] Input device 830 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the vehicle system. Output device 840 may include display devices such as a display screen.

[0142] The present invention also provides a computer-readable storage medium storing a computer program executable by a vehicle-mounted system, wherein when the computer program is run on the vehicle-mounted system, the vehicle-mounted system performs the steps of a voice command feedback method.

[0143] Specifically, the computer storage medium in this embodiment of the invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be—but is not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0144] The present invention also provides a vehicle equipped with the voice command feedback device as described above.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voice command feedback method, characterized by, The voice instruction feedback method comprises: inputting a voice instruction into a large model; pre-judging, according to feedback text of the large model, execution of the voice instruction, and background services and background data involved; pre-loading the background services and the background data.

2. The method of claim 1, wherein, The pre-judging, according to the feedback text of the large model, execution of the voice instruction, and the background services and the background data involved, comprises: acquiring background service keywords in the feedback text; counting word frequencies of the background service keywords; acquiring background service keywords in the top N positions in the word frequency counting; determining, as background services and background data associated with execution of the voice instruction, background services and background data associated with the top N background service keywords.

3. The method of claim 1, wherein, Further comprising: intelligently correcting received incomplete, ambiguous or fuzzy voice instructions.

4. The method of claim 3, wherein, The intelligently correcting received incomplete, ambiguous or fuzzy voice instructions comprises: acquiring a context of the voice instruction; correcting the voice instruction according to analysis of the context.

5. The method of claim 3, wherein, The intelligently correcting received incomplete, ambiguous or fuzzy voice instructions comprises: acquiring voice instructions similar to a current voice instruction; correcting the current voice instruction using the similar voice instructions.

6. The method of claim 1, wherein, Further comprising: preliminarily feeding back the executed voice instruction.

7. The method of claim 1, wherein, The preliminarily feeding back the executed voice instruction comprises: preliminarily feeding back the executed voice instruction according to a current system state.

8. A voice command feedback device, characterized by The voice instruction feedback device comprises: an input module configured to input a voice instruction into a large model; a pre-judging module configured to pre-judge, according to feedback text of the large model, execution of the voice instruction, and background services and background data involved; a pre-loading module configured to pre-load the background services and the background data.

9. A car kit, characterized by The car machine comprises a processor and a memory, the memory stores a computer program, the computer program is loaded and executed by the processor to realize the voice instruction feedback method as claimed in any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the voice instruction feedback method as claimed in any one of claims 1 to 7.

Images