Vehicle voice control method and device, vehicle and storage medium

Abstract

The invention relates to the technical field of vehicles, in particular to a vehicle voice control method and device, a vehicle and a storage medium, and the method comprises the steps: collecting voice information of at least one driver and passenger in a vehicle cabin; performing message text rewriting on the voice information by using the user vehicle control intention recognition model to obtain a rewritten message text; based on an information retrieval enhancement technology, determining a target similarity text of the rewritten message text, inputting the rewritten message text and the target similarity text into a user vehicle control intention recognition model to recognize a voice control intention of a driver and a passenger, and generating a voice control instruction of a vehicle according to the voice control intention, and executing a control action corresponding to the voice control instruction. Therefore, the problems that in the related technology, due to the fact that a large number of rules and a large number of mapping functions are written, voice recognition is limited, due to too many rules, voice recognition is disordered, and the accuracy and intelligence of vehicle voice control are reduced are solved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, vehicle and storage medium for vehicle voice control. Background Art

[0002] The world is in a period of accelerated transformation from industrial economy to digital economy, and my country's economy is also in a critical stage of kinetic energy conversion. The automobile industry is an important pillar industry of the national economy and plays an important role in the national economic and social development. Accurately understanding and grasping the connotation and significance of digital transformation will provide an important engine for the high-quality development of the automobile industry. Intelligent cars and intelligent car cockpits, for example, the map navigation, air conditioning settings, doors and windows, seat adjustments, etc. of the smart cockpit can all be controlled by voice, which is crucial.

[0003] The vehicle voice control technology in the related technology is mainly divided into four links: speech to text ASR (Automatic Speech Recognition) - a large number of rules to recognize instructions - instructions are sent to vehicle control - TTS (Text To Speech). The existing method of the second link of vehicle voice control technology is mainly to write a large number of rules and a large number of mapping functions, that is, to perform voice control on the vehicle's air conditioning settings, doors and windows, seat adjustment, etc. by writing a large number of rules and a large number of mapping function APIs.

[0004] However, the related technology requires writing a large number of rules and mapping functions, which leads to limitations in speech recognition and low efficiency. In addition, too many rules can easily cause confusion in speech recognition, reduce the accuracy and intelligence of vehicle voice control, and reduce the user's driving experience, which needs to be solved urgently. Summary of the invention

[0005] The present application provides a method, device, vehicle and storage medium for vehicle voice control to solve the problem in the related art that a large number of rules and a large number of mapping functions are written, resulting in limitations in voice recognition and low efficiency of voice recognition, and too many rules easily cause confusion in voice recognition, reduce the accuracy and intelligence of vehicle voice control, and reduce the driving experience of users.

[0006] The first aspect of the present application provides a method for vehicle voice control, comprising the following steps: collecting voice information of at least one driver and passenger in a vehicle cabin; rewriting the voice information into a message text using a large model for user vehicle control intention recognition to obtain a rewritten message text; based on information retrieval enhancement technology, determining a target similarity text for the rewritten message text, and inputting the rewritten message text and the target similarity text into the user vehicle control intention recognition model to identify the voice control intention of the at least one driver and passenger, and generating a voice control instruction for the vehicle according to the voice control intention to execute a control action corresponding to the voice control instruction.

[0007] Optionally, in one embodiment of the present application, before using the user vehicle control intention recognition model to rewrite the voice information as a message text, it also includes: converting the voice information to obtain voice information in a preset format; inputting the voice information in the preset format into a preset language model, and training the preset language model using a preset fine-tuning algorithm and a preset framework to obtain the user vehicle control intention recognition model.

[0008] Optionally, in one embodiment of the present application, before determining the target similarity text of the rewritten message text based on information retrieval enhancement technology, it also includes: obtaining a vehicle control instruction document of the vehicle; cutting and processing the vehicle control instruction document to obtain multiple cut partial documents; vectorizing the multiple partial documents using a preset text vector model to obtain vectorized text; and storing the vectorized text in a target vector library to obtain an updated vector library.

[0009] Optionally, in one embodiment of the present application, determining the target similarity text of the rewritten message text based on information retrieval enhancement technology includes: using information retrieval enhancement technology to determine the text vector in the updated vector library that meets the preset similarity condition; based on the text vector that meets the preset similarity condition, determining the target similarity text of the rewritten message text.

[0010] Optionally, in one embodiment of the present application, after executing the control action corresponding to the control instruction, it also includes: judging whether the vehicle has turned on the voice control completion reminder, wherein the voice control completion reminder method includes a voice reminder method and a pop-up screen reminder method; when the vehicle turns on the voice control completion reminder, the voice control completion reminder is sent to a preset terminal.

[0011] The second aspect of the present application provides a vehicle voice control device, including: a collection module for collecting voice information of at least one driver and passenger in a vehicle cabin; an acquisition module for rewriting the voice information into a message text using a large model for user vehicle control intention recognition to obtain a rewritten message text; a generation module for determining a target similarity text of the rewritten message text based on information retrieval enhancement technology, and inputting the rewritten message text and the target similarity text into the user vehicle control intention recognition model to identify the voice control intention of the at least one driver and passenger, and generating a voice control instruction for the vehicle according to the voice control intention to execute a control action corresponding to the voice control instruction.

[0012] Optionally, in one embodiment of the present application, the device of the embodiment of the present application also includes: a first acquisition unit, which is used to convert the voice information to obtain voice information in a preset format before rewriting the voice information into message text using a large model of user vehicle control intention recognition; a second acquisition unit, which is used to input the voice information in a preset format into a preset language model before rewriting the voice information into message text using a large model of user vehicle control intention recognition, and train the preset language model using a preset fine-tuning algorithm and a preset framework to obtain the user vehicle control intention recognition model.

[0013] Optionally, in one embodiment of the present application, the device of the embodiment of the present application also includes: a first determination module, used to obtain a vehicle control instruction document of the vehicle before determining the target similarity text of the rewritten message text based on information retrieval enhancement technology; a first processing module, used to cut the vehicle control instruction document for cutting processing to obtain multiple cut partial documents before determining the target similarity text of the rewritten message text based on information retrieval enhancement technology; a second processing module, used to vectorize the multiple partial documents using a preset text vector model to obtain vectorized text before determining the target similarity text of the rewritten message text based on information retrieval enhancement technology; a second determination module, used to store the vectorized text in a target vector library before determining the target similarity text of the rewritten message text based on information retrieval enhancement technology to obtain an updated vector library.

[0014] Optionally, in one embodiment of the present application, the generation module includes: a first determination unit, used to determine the text vectors in the updated vector library that meet the preset similarity conditions by using information retrieval enhancement technology; a second determination unit, used to determine the target similarity text of the rewritten message text based on the text vectors that meet the preset similarity conditions.

[0015] Optionally, in one embodiment of the present application, the device of the embodiment of the present application also includes: a judgment module, used to judge whether the vehicle has turned on the voice control completion reminder after executing the control action corresponding to the control instruction, wherein the voice control completion reminder method includes a voice reminder method and a pop-up screen reminder method; a sending module, used to send the voice control completion reminder to a preset terminal after executing the control action corresponding to the control instruction, when the vehicle turns on the voice control completion reminder.

[0016] A third aspect of the present application provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle voice control method as described in the above embodiment.

[0017] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program that implements the above vehicle voice control method when executed by a processor.

[0018] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above vehicle voice control method.

[0019] The embodiment of the present application can use the user vehicle control intention recognition model to rewrite the voice information to obtain the rewritten message text, then determine the target similarity text of the rewritten message text based on the information retrieval enhancement technology, and input the rewritten message text and the target similarity text into the user vehicle control intention recognition model to recognize the voice control intention of at least one driver and passenger, and generate the vehicle's voice control instructions according to the voice control intention to execute the control action corresponding to the voice control instruction, effectively improving the accuracy and intelligence of the vehicle's voice control. Thus, the problems in the related art of writing a large number of rules and a large number of mapping functions, resulting in limitations in voice recognition and low efficiency in voice recognition, and too many rules, which easily cause confusion in voice recognition and reduce the accuracy and intelligence of vehicle voice control, are solved.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1A flow chart of a vehicle voice control method provided according to an embodiment of the present application;

[0023] Figure 2 This is a schematic diagram of fine-tuning a large model of a vehicle control intelligent body according to a specific embodiment of the present application;

[0024] Figure 3 This is a schematic diagram of information retrieval enhancement and prompt implementation of a large model of a vehicle control intelligent body in a specific embodiment of the present application;

[0025] Figure 4 This is a flowchart of the information retrieval enhancement technology of a specific embodiment of the present application;

[0026] Figure 5 This is a schematic diagram of text similarity calculation for a specific embodiment of the present application;

[0027] Figure 6 A schematic diagram of a vehicle voice control link according to a specific embodiment of the present application;

[0028] Figure 7 A schematic diagram of the structure of a vehicle voice control device provided according to an embodiment of the present application;

[0029] Figure 8 It is a schematic diagram of the structure of a vehicle provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0031] The following describes the method, device, vehicle and storage medium of the vehicle voice control of the embodiment of the present application with reference to the accompanying drawings. In view of the problems that a large number of rules and a large number of mapping functions are written in the related art mentioned in the above background technology, which leads to the limitation of voice recognition and low efficiency of voice recognition, and too many rules easily cause confusion in voice recognition, reduce the accuracy and intelligence of vehicle voice control, etc., the present application provides a method of vehicle voice control, in which the voice information can be rewritten according to the user vehicle control intention recognition model to obtain the rewritten message text, then, based on the information retrieval enhancement technology, the target similarity text of the rewritten message text is determined, and the rewritten message text and the target similarity text are input into the user vehicle control intention recognition model to identify the voice control intention of the driver and passenger, and generate the voice control instruction of the vehicle according to the voice control intention to execute the control action corresponding to the voice control instruction, effectively improving the accuracy and intelligence of vehicle voice control. Thus, the problems that a large number of rules and a large number of mapping functions are written in the related art, leading to the limitation of voice recognition and low efficiency of voice recognition, and too many rules easily cause confusion in voice recognition, reduce the accuracy and intelligence of vehicle voice control, etc.

[0032] Specifically, Figure 1 A flow chart of a vehicle voice control method provided in an embodiment of the present application.

[0033] like Figure 1 As shown, the vehicle voice control method includes the following steps:

[0034] In step S101, voice information of at least one driver and passenger in a vehicle cabin is collected.

[0035] It can be understood that the embodiments of the present application can collect voice information of at least one driver and passenger in the vehicle cabin. For example, the present application can utilize a microphone or other recording device installed inside the vehicle to capture the voice of the driver and passenger. The microphone can be single or multiple and distributed in different positions in the cabin to ensure the comprehensiveness and clarity of sound capture. In addition, the collection of voice information can be continuous or triggered, that is, recording only starts when a wake-up word or a specific command is detected, which effectively improves the feasibility of vehicle voice control.

[0036] In step S102, the voice information is rewritten using the user vehicle control intention recognition model to obtain a rewritten message text.

[0037] It is understandable that the embodiment of the present application can train a pre-set language model according to voice information to obtain a user vehicle control intention recognition model, for example, Figure 2As shown, by fine-tuning the open source big model llama based on the collected voice information, the open source big model llama becomes a big model for user vehicle control intention recognition in the vehicle field. For example, the new model can be named CM big model, and the user message text is rewritten through this model to make the semantic intention of the rewritten user message clearer. For example, "The weather is hot" can be rewritten as "Turn on the air conditioner" or "Play a song" can be rewritten as "Turn on the music", which effectively improves the accuracy of voice control command generation.

[0038] Among them, in one embodiment of the present application, before using the user vehicle control intention recognition model to rewrite the voice information as a message text, it also includes: converting the voice information to obtain voice information in a preset format; inputting the voice information in the preset format into a preset language model, and training the preset language model using a preset fine-tuning algorithm and a preset framework to obtain a user vehicle control intention recognition model.

[0039] For example, the embodiment of the present application can convert voice information into json format, and input the voice information in json format into the open source big model llama, and use the fine-tuning algorithm lora and the fine-tuning framework LLaMA factory to make the open source big model llama a user vehicle control intention recognition big model in the vehicle field. The new model can be named CM big model, which effectively improves the robustness of vehicle voice control.

[0040] In the actual implementation process, the basic principle of the lora fine-tuning algorithm is to freeze the pre-trained model parameters, and then add a trainable bypass matrix (low-rank separable matrix) to each layer of the Transfomer. Then, the bypass output is added to the initial path output and input into the network, and only these newly added bypass matrix parameters are trained; among them, the low-rank separable matrix is ​​composed of two matrices, the first matrix is ​​responsible for dimensionality reduction, and the second matrix is ​​responsible for dimensionality increase. The dimension of the middle layer is r, so as to simulate the intrinsic rank. These two low-rank matrices can greatly reduce the number of parameters; in addition, the LLaMAfactory framework is a relatively complete encapsulation of LLM fine-tuning tools, which can help users quickly train and fine-tune most LLM models. After the fine-tuning model, reinforcement learning can be carried out, and PPO or DPO algorithms can be used for reinforcement alignment learning, which effectively improves the generalization ability of the model.

[0041] In step S103, based on the information retrieval enhancement technology, the target similarity text of the rewritten message text is determined, and the rewritten message text and the target similarity text are input into the user vehicle control intention recognition model to identify the voice control intention of at least one driver and passenger, and generate a voice control instruction of the vehicle according to the voice control intention to execute the control action corresponding to the voice control instruction.

[0042] It can be understood that the embodiment of the present application can determine the target similarity text of the rewritten message text based on the information retrieval enhancement technology. For example, the rewritten user message text can be vectorized, and the text similarity retrieval can be performed through the RAG (Retrieval-augmented Generation, information retrieval enhancement) technology to output the topk similar texts, and then perform fine sorting to output the text with the highest similarity, as shown in Table 1 and Table 2, wherein Table 1 is a partial sample of air conditioning control and the corresponding format requirement table, and Table 1 is as follows:

[0043] Table 1

[0044]

[0045] Table 2 is a table of some examples of music APP control and the corresponding format requirements. The specific table 2 is as follows:

[0046] Table 2

[0047]

[0048] Then, if Figure 3 As shown, the embodiment of the present application can generate the vehicle control command format through the Prompt project, combine the rewritten message text and the text retrieved by similarity into a Prompt, and input it into the CM large model, so as to output the specified content and format, such as the voice control command in json format, and then the generated vehicle control command can be sent to the vehicle equipment through Action, and the parameter format is spliced ​​according to the rules to form a vehicle control command interface {CarInfoApiproxy.getInstance().setConditioner.acSwitch,0}; in order to generate a form that the vehicle control can understand, the intelligence level of the vehicle is effectively improved to meet the driving needs of users.

[0049] In an embodiment of the present application, the present application can perform voice control on the vehicle's air conditioning settings, door and window settings, seat adjustment, etc., to help users quickly set vehicle functions (for example, air conditioning temperature settings, vehicle sunroof switch settings, etc.), making the cockpit more intelligent and improving the safety of users when using the car.

[0050] Optionally, in one embodiment of the present application, before determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology, it also includes: obtaining a vehicle control instruction document of the vehicle; cutting the vehicle control instruction document for cutting processing to obtain multiple cut partial documents; using a preset text vector model to vectorize the multiple partial documents to obtain vectorized text; storing the vectorized text in a target vector library to obtain an updated vector library.

[0051] As a possible way to achieve this, Figure 3 As shown, the embodiment of the present application can sort out the vehicle control API instruction documents and perform information retrieval enhancement technology. First, the embodiment of the present application can cut the document into chunks, and then use the M3E model to vectorize the chunks. The vectorized text is stored in the faiss library to obtain an updated vector library, which effectively improves the accuracy of the data.

[0052] Among them, in one embodiment of the present application, based on the information retrieval enhancement technology, the target similarity text of the rewritten message text is determined, including: using the information retrieval enhancement technology to determine the text vector that meets the preset similarity condition in the updated vector library; based on the text vector that meets the preset similarity condition, determine the target similarity text of the rewritten message text.

[0053] For example, the embodiment of the present application can use information retrieval enhancement technology to perform text similarity search, find the topk most similar vectors, and then perform fine sorting to finally output a text vector with the highest similarity to the user message. Then, the output similar text vector and the text message rewritten by the user are put into the prompt, and then the CM large model is called to output the instruction parameters, for example, [{"name":"Set rear seat ventilation level","method":"seat","type":"set","parameters":{"rearRightVentilation":1}}], which effectively improves the accuracy of voice control command generation.

[0054] It should be noted that if Figure 4As shown in the figure, it is a flowchart of information retrieval enhancement technology. The retrieval enhancement generation model combines language model and information retrieval technology. Specifically, when the model needs to generate text or answer questions, it will first retrieve relevant information from a large document collection, and then use the retrieved information to guide the generation of text, thereby improving the quality and accuracy of prediction. The theoretical basis of the information retrieval enhancement model mainly revolves around the use of retrieval, knowledge filling, intelligent assistant, information retrieval, data update, customization ability, security management, etc., to provide accurate, timely, highly explanatory, highly customized, and secure services. The technical steps are to first perform text similarity search, find the first n similar text vectors, and then finely sort them, and finally output the most similar text vector, put the output of the most similar text vector and the user's new text message into the prompt, and then call the CM large model.

[0055] like Figure 5 The figure shows the text similarity calculation graph, i.e., the Euclidean distance of text similarity retrieval. Euclidean distance is a distance measurement method used to measure the straight-line distance between two points in a multidimensional space. The most common and intuitive distance measurement method is usually used in numerical data or feature space. The principle of Euclidean distance can be explained as follows: Suppose there are two points A and B, whose coordinates in two-dimensional space are (x1, y1) and (x2, y2) respectively. To calculate the distance between these two points, the following formula can be used:

[0056]

[0057] This formula actually calculates the straight-line distance between point A and point B. Extending this concept to multidimensional space, if there are two points A and B, their coordinates in n-dimensional space are (x1, y1, z1, ..., w n ) and (x2,y2,z2,...,w n ), then the Euclidean distance can be expressed as:

[0058]

[0059] Optionally, in one embodiment of the present application, after executing the control action corresponding to the control instruction, it also includes: determining whether the vehicle has turned on the voice control completion reminder, wherein the voice control completion reminder method includes a voice reminder method and a pop-up screen reminder method; when the vehicle turns on the voice control completion reminder, the voice control completion reminder is sent to a preset terminal.

[0060] In some embodiments, the embodiments of the present application can determine whether the vehicle has turned on voice control to complete the reminder. For example, the user can turn on the reminder through a physical button. When the vehicle turns on voice control to complete the reminder, a reply can be made through the CM large model Agent (including text-to-speech technology TTS), such as "The air conditioner is on", and converted into voice form, which effectively improves the interactivity of the vehicle.

[0061] For example, Figure 6 As shown, this is my logic diagram of vehicle voice control. First, the user speaks out the command requirements for the vehicle through voice, for example, "The weather is so hot." The embodiment of the present application can convert speech to text through ASR, and then rewrite the user text message through the CM large model to make the semantics clearer. "The weather is so hot" becomes "Turn on the air conditioner." The rewritten user text message is searched by RAG, and then the parameters are output through prompt, and the API content is associated. The format is output and the command is issued through Action. The reply is made through Agent text-to-speech, which effectively improves the accuracy and intelligence of vehicle voice control.

[0062] According to the vehicle voice control method proposed in the embodiment of the present application, the voice information can be rewritten according to the user vehicle control intention recognition model to obtain the rewritten message text, and then, based on the information retrieval enhancement technology, the target similarity text of the rewritten message text is determined, and the rewritten message text and the target similarity text are input into the user vehicle control intention recognition model to recognize the voice control intention of the driver and passenger, and generate the vehicle's voice control instructions according to the voice control intention to execute the control action corresponding to the voice control instruction, which effectively improves the accuracy and intelligence of the vehicle voice control. As a result, the problems of writing a large number of rules and a large number of mapping functions in the related technology, resulting in limitations in voice recognition and low efficiency in voice recognition, and too many rules easily causing confusion in voice recognition, reducing the accuracy and intelligence of vehicle voice control, etc.

[0063] Next, a vehicle voice control device according to an embodiment of the present application is described with reference to the accompanying drawings.

[0064] Figure 7 It is a block diagram of a vehicle voice control device according to an embodiment of the present application.

[0065] like Figure 7 As shown, the vehicle voice control device 10 includes: a collection module 100, an acquisition module 200 and a generation module 300.

[0066] Specifically, the collection module 100 is used to collect voice information of at least one driver and passenger in the vehicle cabin.

[0067] The acquisition module 200 is used to rewrite the voice information into a message text by using the user vehicle control intention recognition large model to obtain the rewritten message text.

[0068] The generation module 300 is used to determine the target similarity text of the rewritten message text based on the information retrieval enhancement technology, and input the rewritten message text and the target similarity text into the user vehicle control intention recognition model to identify the voice control intention of at least one driver and passenger, and generate a voice control instruction of the vehicle according to the voice control intention to execute the control action corresponding to the voice control instruction.

[0069] Optionally, in one embodiment of the present application, the device 10 of the embodiment of the present application includes: a first acquisition unit and a second acquisition unit.

[0070] Among them, the first acquisition unit is used to convert the voice information before rewriting the message text of the voice information using the user vehicle control intention recognition model to obtain the voice information in a preset format.

[0071] The second acquisition unit is used to input the voice information in a preset format into a preset language model before rewriting the message text of the voice information using the user vehicle control intention recognition model, and train the preset language model using a preset fine-tuning algorithm and a preset framework to obtain a user vehicle control intention recognition model.

[0072] Optionally, in one embodiment of the present application, the device 10 of the embodiment of the present application further includes: a first determination module, a first processing module, a second processing module and a second determination module.

[0073] Among them, the first determination module is used to obtain the vehicle control instruction document of the vehicle before determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology.

[0074] The first processing module is used to cut and process the vehicle control instruction document before determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology to obtain a plurality of cut partial documents.

[0075] The second processing module is used to perform vectorization processing on multiple partial documents using a preset text vector model to obtain vectorized text before determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology.

[0076] The second determination module is used to store the vectorized text in a target vector library before determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology to obtain an updated vector library.

[0077] Optionally, in one embodiment of the present application, the generating module 300 includes: a first determining unit and a second determining unit.

[0078] The first determination unit is used to determine the text vectors that meet the preset similarity condition in the updated vector library by using the information retrieval enhancement technology.

[0079] The second determining unit is used to determine a target similarity text of the rewritten message text based on the text vectors that meet the preset similarity condition.

[0080] Optionally, in an embodiment of the present application, the device 10 of the embodiment of the present application further includes: a judgment module and a sending module.

[0081] Among them, the judgment module is used to judge whether the vehicle turns on the voice control completion reminder after executing the control action corresponding to the control instruction, wherein the voice control completion reminder method includes a voice reminder method and a pop-up screen reminder method.

[0082] The sending module is used to send a voice control completion reminder to a preset terminal after executing the control action corresponding to the control instruction and when the vehicle turns on the voice control completion reminder.

[0083] It should be noted that the above explanation of the vehicle voice control method embodiment is also applicable to the vehicle voice control device of this embodiment, and will not be repeated here.

[0084] According to the vehicle voice control device proposed in the embodiment of the present application, the voice information can be rewritten using the user vehicle control intention recognition model to obtain the rewritten message text, and then, based on the information retrieval enhancement technology, the target similarity text of the rewritten message text is determined, and the rewritten message text and the target similarity text are input into the user vehicle control intention recognition model to identify the voice control intention of at least one driver and passenger, and generate the vehicle's voice control instructions according to the voice control intention to execute the control action corresponding to the voice control instruction, effectively improving the accuracy and intelligence of the vehicle voice control. Thus, the problems in the related art of writing a large number of rules and a large number of mapping functions, resulting in limitations in voice recognition and low efficiency in voice recognition, and too many rules, which easily cause confusion in voice recognition and reduce the accuracy and intelligence of vehicle voice control, are solved.

[0085] Figure 8 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle may include:

[0086] A memory 801 , a processor 802 , and a computer program stored in the memory 801 and executable on the processor 802 .

[0087] When the processor 802 executes the program, the vehicle voice control method provided in the above embodiment is implemented.

[0088] Furthermore, the vehicle also includes:

[0089] The communication interface 803 is used for communication between the memory 801 and the processor 802 .

[0090] The memory 801 is used to store computer programs that can be executed on the processor 802 .

[0091] The memory 801 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0092] If the memory 801, the processor 802 and the communication interface 803 are implemented independently, the communication interface 803, the memory 801 and the processor 802 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0093] Optionally, in a specific implementation, if the memory 801, the processor 802 and the communication interface 803 are integrated on a chip, the memory 801, the processor 802 and the communication interface 803 can communicate with each other through an internal interface.

[0094] The processor 802 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0095] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above vehicle voice control method is implemented.

[0096] This embodiment also provides a computer program product, including a computer program, which is used to implement the above vehicle voice control method when executed.

[0097] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0098] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0099] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0100] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0101] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0102] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0104] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for vehicle voice control, characterized in that: The following steps are involved: Collecting voice information of at least one driver or passenger in the vehicle cabin; Rewriting the voice information into a message text using a large model for recognizing the user's vehicle control intention, so as to obtain a rewritten message text; Based on information retrieval enhancement technology, the target similarity text of the rewritten message text is determined, and the rewritten message text and the target similarity text are input into the user vehicle control intention recognition model to identify the voice control intention of at least one driver and passenger, and generate a voice control instruction of the vehicle according to the voice control intention to execute the control action corresponding to the voice control instruction.

2. The method according to claim 1, characterized in that Before rewriting the voice information into a message text using the user vehicle control intention recognition model, the method further includes: Converting the voice information to obtain voice information in a preset format; The voice information in the preset format is input into a preset language model, and the preset language model is trained using a preset fine-tuning algorithm and a preset framework to obtain the user vehicle control intention recognition model.

3. The method according to claim 1, characterized in that Before determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology, the method further includes: Obtaining a vehicle control command document of the vehicle; Cutting the vehicle control instruction document to obtain a plurality of cut partial documents; Performing vectorization processing on the plurality of partial documents using a preset text vector model to obtain vectorized texts; The vectorized text is stored in a target vector library to obtain an updated vector library.

4. The method according to claim 3, characterized in that The step of determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology includes: Determine the text vectors in the updated vector library that meet the preset similarity condition by using information retrieval enhancement technology; Based on the text vectors satisfying the preset similarity condition, the target similarity text of the rewritten message text is determined.

5. The method according to claim 1, characterized in that After executing the control action corresponding to the control instruction, the method further includes: Determine whether the vehicle turns on the voice control completion reminder, wherein the voice control completion reminder mode includes a voice reminder mode and a pop-up screen reminder mode; When the vehicle turns on the voice control completion reminder, the voice control completion reminder is sent to a preset terminal.

6. A vehicle voice control device, characterized in that: include: A collection module, used to collect voice information of at least one driver and passenger in the vehicle cabin; An acquisition module, used for rewriting the voice information into a message text by using a large model for recognizing the user's vehicle control intention, so as to obtain a rewritten message text; A generation module is used to determine the target similarity text of the rewritten message text based on information retrieval enhancement technology, and input the rewritten message text and the target similarity text into the user vehicle control intention recognition model to identify the voice control intention of at least one driver and passenger, and generate a voice control instruction of the vehicle according to the voice control intention to execute the control action corresponding to the voice control instruction.

7. The device according to claim 6, characterized in that Also includes: A first acquisition unit is used to convert the voice information to obtain the voice information in a preset format before rewriting the voice information into a message text using the user vehicle control intention recognition model; The second acquisition unit is used to input the voice information in a preset format into a preset language model before rewriting the voice information into a message text using the user vehicle control intention recognition model, and train the preset language model using a preset fine-tuning algorithm and a preset framework to obtain the user vehicle control intention recognition model.

8. The device according to claim 6, characterized in that Also includes: A first determination module is used to obtain a vehicle control instruction document of the vehicle before determining a target similarity text of the rewritten message text based on the information retrieval enhancement technology; A first processing module is used for cutting and processing the vehicle control instruction document to obtain a plurality of cut partial documents before determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology; A second processing module is used to perform vectorization processing on the plurality of partial documents using a preset text vector model to obtain vectorized texts before determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology; The second determination module is used to store the vectorized text in a target vector library before determining the target similarity text of the rewritten message text based on the information retrieval enhancement technology to obtain an updated vector library.

9. A vehicle, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle voice control method as described in any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the vehicle voice control method as described in any one of claims 1 to 5.

Images