Model training method, vehicle control method, device, equipment, medium and product

Abstract

The invention relates to a model training method, a vehicle control method, a device, equipment, a medium and a product, and the method comprises the steps: generating a plurality of second control instructions according to the scene information (sample driving information and sample user information) of a sample vehicle, a first control instruction and the powerful common sense reasoning capability of a large model; the scene information and the first control instruction serve as training samples, the multiple second control instructions serve as labels, model training is carried out, and a control instruction generation model is generated; and generating the capabilities of a plurality of target control instructions associated with the functions corresponding to the initial control instruction. In practical application, only one control instruction is needed, some vehicle control operations can be completed, equipment which is not known by a user in the vehicle can be controlled, and the efficiency and comprehensiveness of vehicle control are improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, specifically to a model training method, a vehicle control method, a device, equipment, medium, and product. Background Technology

[0002] With the widespread adoption of voice assistants in smartphones and smart speakers, voice interaction technology has developed rapidly. These technologies have gradually been applied to the automotive field, forming in-cabin voice assistants.

[0003] A cockpit voice assistant is an intelligent voice interaction technology integrated into a car's cockpit system. It allows drivers and passengers to control various vehicle functions via voice commands, such as navigation, music playback, telephone communication, and vehicle settings. Users can actively speak a voice command, and the vehicle will then execute it exactly as spoken.

[0004] However, existing methods for controlling vehicles using cockpit voice assistants are not very efficient or comprehensive. Summary of the Invention

[0005] One objective of this invention is to provide a model training method and a vehicle control method to solve the problems of low efficiency and comprehensiveness in vehicle control in the prior art; a second objective is to provide a model training device and a vehicle control device; a third objective is to provide an electronic device; a fourth objective is to provide a computer-readable storage medium; a fifth objective is to provide a computer program product; and a sixth objective is to provide a vehicle.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A model training method, comprising:

[0008] The system acquires a first control command, sample driving information of the sample vehicle, and sample user information of the driver / rider of the sample vehicle. The first control command is used to control the first sample device in the sample vehicle.

[0009] Based on the sample driving information, the sample user information, and the first control command, a prompt message is generated;

[0010] The prompt is input into the large model to obtain multiple second control commands output by the large model. The multiple second control commands are used to control the first sample device.

[0011] Based on the sample driving information, the sample user information, the first control command, and the multiple second control commands, a model is trained to obtain a control command generation model.

[0012] Based on the aforementioned technical methods, multiple second control commands are generated using the sample vehicle's scene information (sample driving information and sample user information), the first control command, and the powerful common-sense reasoning ability of the large model. Then, the scene information and the first control command are used as training samples, and the multiple second control commands are used as labels for model training to generate a control command generation model. This model has the ability to generate multiple target control commands associated with the functions corresponding to the initial control command, based on the initial control command and the current scene. In other words, only one initial control command needs to be input to generate a series of target control commands, enabling control of devices in the vehicle that the user is unaware of, thus improving the efficiency and comprehensiveness of vehicle control.

[0013] Furthermore, the step of inputting the prompt into the large model and obtaining multiple second control commands output by the large model includes:

[0014] Input the prompt into the large model, and obtain the multiple second control commands and corresponding response messages output by the large model;

[0015] Accordingly, the step of training a model based on the sample driving information, the sample user information, the first control command, and the plurality of second control commands to obtain a control command generation model includes:

[0016] The model is trained based on the sample driving information, the sample user information, the first control command, the multiple second control commands, and the response, to obtain the control command generation model.

[0017] Based on the aforementioned technical means, during subsequent model training using these multiple second control commands, the trained control command generation model can output the corresponding target response. By playing the generated target response after executing the target control command generated by the control command generation model, the user's driving experience is effectively improved.

[0018] Furthermore, the step of training a model based on the sample driving information, the sample user information, the first control command, the plurality of second control commands, and the response to obtain a control command generation model includes:

[0019] Generate an instruction identifier based on the plurality of second control instructions and their corresponding response statements;

[0020] The control command generation model is obtained by training the model based on the sample driving information, the sample user information, the first control command, and the command identifier.

[0021] Based on the aforementioned technical methods, by establishing a first mapping relationship among the response, control command, and command identifier, the command identifier can be used as a label for training samples to participate in model training. This effectively reduces the amount of data processing during model training and the amount of data output from the control command generation model during the inference phase, thereby improving model training efficiency and model inference efficiency. During model inference, based on the target command index generated by the control command generation model and the aforementioned first mapping relationship, the corresponding target control command and target response can be obtained.

[0022] Furthermore, the step of inputting the prompt into the large model and obtaining multiple second control commands output by the large model includes:

[0023] Input the prompt into the large model to obtain multiple sample control instruction groups output by the large model. Each sample control instruction group includes multiple third control instructions and corresponding response messages.

[0024] Accordingly, the step of training a model based on the sample driving information, the sample user information, the first control command, and the plurality of second control commands to obtain a control command generation model includes:

[0025] The control command generation model is obtained by training the model based on the sample driving information, the sample user information, the first control command, and the multiple sample control command groups.

[0026] Based on the above technical means, multiple sample control command groups are generated through a large model, so that multiple sample control command groups can be used as labels for model training, thereby improving the richness of target control commands generated by the control command generation model during model inference.

[0027] Furthermore, the sample driving information includes at least one of the following: navigation data, weather data, and energy data;

[0028] The sample user information includes at least one of the following: the number of users driving the sample vehicle, the age, gender, expression, actions, and position of each user in the sample vehicle, and the position of the speaking user in the sample vehicle.

[0029] Furthermore, the plurality of second control commands are also used to control a second sample device, which is a device in the sample vehicle that is associated with the first sample device in performing functions.

[0030] Based on the aforementioned technical means, after subsequent model training based on these multiple second control commands, the trained control command generation model can generate multiple target control commands with similar functions to the initial control commands. This allows for the invocation of multiple devices or the configuration of multiple items on the same device while meeting user needs. This solves the problem of some devices being idle due to users' lack of understanding of the full range of vehicle functions, thereby improving the comprehensiveness of vehicle control.

[0031] A vehicle control method, the method comprising:

[0032] After the target user driving the target vehicle initiates an initial control command, the driving information of the target vehicle and the information of the user driving the target vehicle are obtained. The initial control command is used to control the first device in the target vehicle.

[0033] The driving information, the user information, and the initial control command are input into the control command generation model to obtain multiple target control commands. The multiple target control commands are used to control the first device. The control command generation model is obtained by training the model based on the above model training method.

[0034] The multiple target control commands are executed to control the target vehicle.

[0035] Based on the aforementioned technical means, after the user inputs the initial control command, multiple target control commands that can be linked with the initial control command can be determined by combining the current scenario of the target vehicle (driving information and user information). Users no longer need to manually input multiple control commands; a single control command is sufficient to complete various vehicle control operations. This evolves from the existing "point-and-shoot" approach to a more comprehensive and collaborative system that assists users in controlling in-vehicle devices and systems, improving the efficiency and comprehensiveness of vehicle control and providing users with a more nuanced and attentive technological experience.

[0036] Furthermore, the step of inputting the driving information, the user information, and the initial control command into the control command generation model to obtain multiple target control commands includes:

[0037] The driving information, the user information, and the initial control command are input into the control command generation model to obtain the target command identifier;

[0038] Obtain the plurality of target control instructions corresponding to the target instruction identifier.

[0039] Based on the above technical means, using the instruction identifier as the output of the control instruction generation model effectively reduces the amount of data processing during model training and the amount of data output of the control instruction generation model during the inference stage, thereby improving the model training efficiency and model inference efficiency.

[0040] Furthermore, after obtaining the target instruction identifier, the method further includes:

[0041] Obtain the target response message corresponding to the target instruction identifier;

[0042] Accordingly, after executing the plurality of target control commands to control the target vehicle, the method further includes:

[0043] Play the target response.

[0044] Based on the above technical means, users can be notified that the target control command (modified target control command) has been successfully executed, which effectively improves the user experience.

[0045] Furthermore, executing the plurality of target control commands to control the target vehicle includes:

[0046] Based on pre-stored user preference parameters, the multiple target control commands are modified;

[0047] The modified target control commands are executed to control the target vehicle.

[0048] Based on the above technical means, the personalized needs of users for configuration parameters are effectively taken into account.

[0049] Furthermore, the step of obtaining the driving information of the target vehicle and the user information of the driver / passenger of the target vehicle after the target user initiates the initial control command includes:

[0050] Determine the historical control commands generated by the control command generation model within the historical time period;

[0051] After the target user initiates the initial control command, it is determined whether the initial control command and the historical control command indicate control of the same configuration item of the same device;

[0052] If the initial control command and the historical control command do not indicate control of the same configuration item of the same device, then the driving information of the target vehicle and the user information of the driver of the target vehicle are obtained.

[0053] Based on the above technical means, user feedback is taken into account during the model inference process to avoid causing user resentment by executing control commands to generate target control commands, thus further taking into account the user experience and improving the accuracy of vehicle control.

[0054] Furthermore, the driving information includes at least one of the following: navigation data, weather data, and energy data;

[0055] The user information includes at least one of the following: the number of users driving the target vehicle, the age, gender, expression, actions, and position of each user in the target vehicle, and the position of the target user in the target vehicle.

[0056] Furthermore, the plurality of target control commands are also used to control a second device, which is a device in the target vehicle that is associated with the first device in performing functions.

[0057] Based on the above technical means, multiple target control commands with similar initial control execution functions are generated according to the control command generation model. Multiple devices are invoked while meeting user needs, thereby solving the problem of some devices being idle due to users' lack of understanding of the full range of vehicle functions and improving the comprehensiveness of vehicle control.

[0058] A model training device, comprising:

[0059] The acquisition module is used to acquire a first control command, sample driving information of the sample vehicle, and sample user information of the sample vehicle. The first control command is used to control the first sample device in the sample vehicle.

[0060] The generation module is used to generate a prompt message based on the sample driving information, the sample user information, and the first control command.

[0061] The input module is used to input the prompt into the large model and obtain multiple second control commands output by the large model. The multiple second control commands are used to control the first sample device.

[0062] The training module is used to train a model based on the sample driving information, the sample user information, the first control command, and the plurality of second control commands to obtain a control command generation model.

[0063] Furthermore, the input module is specifically used for:

[0064] Input the prompt into the large model, and obtain the multiple second control commands and corresponding response messages output by the large model;

[0065] Accordingly, the training module is specifically used for:

[0066] The model is trained based on the sample driving information, the sample user information, the first control command, the multiple second control commands, and the response, to obtain the control command generation model.

[0067] Furthermore, the training module is specifically used for:

[0068] Generate an instruction identifier based on the plurality of second control instructions and their corresponding response statements;

[0069] The control command generation model is obtained by training the model based on the sample driving information, the sample user information, the first control command, and the command identifier.

[0070] Furthermore, the input module is specifically used for:

[0071] Input the prompt into the large model to obtain multiple sample control instruction groups output by the large model. Each sample control instruction group includes multiple third control instructions and corresponding response messages.

[0072] Accordingly, the training module is specifically used for:

[0073] The control command generation model is obtained by training the model based on the sample driving information, the sample user information, the first control command, and the multiple sample control command groups.

[0074] Furthermore, the sample driving information includes at least one of the following: navigation data, weather data, and energy data;

[0075] The sample user information includes at least one of the following: the number of users driving the sample vehicle, the age, gender, expression, actions, and position of each user in the sample vehicle, and the position of the speaking user in the sample vehicle.

[0076] Furthermore, the plurality of second control commands are also used to control a second sample device, which is a device in the sample vehicle that is associated with the first sample device in performing functions.

[0077] A vehicle control device, the method comprising:

[0078] The acquisition module is used to acquire the driving information of the target vehicle and the information of the user driving the target vehicle after the target user initiates the initial control command. The initial control command is used to control the first device in the target vehicle.

[0079] The input module is used to input the driving information, the user information, and the initial control command into the control command generation model to obtain multiple target control commands. The multiple target control commands are used to control the first device. The control command generation model is obtained by training the model based on the above model training method.

[0080] The execution module is used to execute the plurality of target control commands to control the target vehicle.

[0081] Furthermore, the input module is specifically used for:

[0082] The driving information, the user information, and the initial control command are input into the control command generation model to obtain the target command identifier;

[0083] Obtain the plurality of target control instructions corresponding to the target instruction identifier.

[0084] Furthermore, after obtaining the target instruction identifier, the acquisition module is also used to:

[0085] Obtain the target response message corresponding to the target instruction identifier;

[0086] Accordingly, after executing the plurality of target control commands to control the target vehicle, the vehicle control device further includes:

[0087] The playback module is used to play the target response.

[0088] Furthermore, the execution module is specifically used for:

[0089] Based on pre-stored user preference parameters, the multiple target control commands are modified;

[0090] The modified target control commands are executed to control the target vehicle.

[0091] Furthermore, the acquisition module is specifically used for:

[0092] Determine the historical control commands generated by the control command generation model within the historical time period;

[0093] After the target user initiates the initial control command, it is determined whether the initial control command and the historical control command indicate control of the same configuration item of the same device;

[0094] If the initial control command and the historical control command do not indicate control of the same configuration item of the same device, then the driving information of the target vehicle and the user information of the driver of the target vehicle are obtained.

[0095] Furthermore, the driving information includes at least one of the following: navigation data, weather data, and energy data;

[0096] The user information includes at least one of the following: the number of users driving the target vehicle, the age, gender, expression, actions, and position of each user in the target vehicle, and the position of the target user in the target vehicle.

[0097] Furthermore, the plurality of target control commands are also used to control a second device, which is a device in the target vehicle that is associated with the first device in performing functions.

[0098] An electronic device includes: a processor, a memory, and computer-executable instructions stored in the memory and executable on the processor, wherein the processor executes the computer-executable instructions to implement the above-described model training method and vehicle control method.

[0099] A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the above-described model training method and vehicle control method.

[0100] A computer program product includes a computer program that, when executed by a processor, is used to implement the aforementioned model training method and vehicle control method.

[0101] A vehicle comprising the aforementioned electronic equipment.

[0102] The beneficial effects of this invention are:

[0103] (1) Users only need to input an initial control command to generate a series of target control commands, evolving from the existing technology of "point and shoot" to "learn by analogy", which improves the efficiency of vehicle control.

[0104] (2) It can control devices in the vehicle that the user has never known, thus improving the comprehensiveness of vehicle control. Attached Figure Description

[0105] Figure 1 A schematic flowchart of Embodiment 1 of the model training method provided in this invention;

[0106] Figure 2 A schematic flowchart of Embodiment 2 of the model training method provided in this invention;

[0107] Figure 3 This is a flowchart illustrating a vehicle control method according to a first embodiment of the present invention.

[0108] Figure 4 This is a flowchart illustrating a second embodiment of the vehicle control method provided by the present invention.

[0109] Figure 5 A flowchart illustrating a third embodiment of the vehicle control method provided in this invention;

[0110] Figure 6 A schematic flowchart of Embodiment 4 of the vehicle control method provided by the present invention;

[0111] Figure 7 A flowchart illustrating Embodiment 5 of the vehicle control method provided by this invention;

[0112] Figure 8 This is a schematic diagram of the structure of the model training device provided in an embodiment of the present invention;

[0113] Figure 9 This is a schematic diagram of the structure of a vehicle control device provided in an embodiment of the present invention;

[0114] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0115] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0116] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0117] Before introducing the embodiments of the present invention, the application background of the embodiments of the present invention will be explained first:

[0118] With the widespread adoption of voice assistants in smartphones and smart speakers, voice interaction technology has developed rapidly. These technologies are gradually being applied to the automotive field, forming in-cabin voice assistants. An in-cabin voice assistant is an intelligent voice interaction technology integrated into the car's cockpit system. It allows drivers and passengers to control various vehicle functions via voice commands, such as navigation, music playback, telephone communication, and vehicle settings. The purpose of this technology is to improve driving safety, reduce driver distraction while operating the vehicle, and provide a more convenient and intuitive user experience.

[0119] Existing in-vehicle voice assistants require the user to actively speak a voice command, after which the vehicle will execute the command exactly as spoken. However, existing in-vehicle voice assistants have the following technical problems:

[0120] 1. The vehicle can only execute a single user command. For example, if the user says "turn on the air conditioning," the vehicle will turn on the air conditioning; if the user says "turn up the air conditioning temperature," the vehicle will turn up the air conditioning temperature. In other words, if the user needs to control multiple devices in the vehicle, or multiple configuration items of the same device, the user needs to issue multiple voice commands, which is cumbersome and inefficient in controlling the car.

[0121] 2. The vehicle has many functions, and it is difficult for users to know all of them, which may result in some functions or devices being idle, making it impossible to guarantee comprehensive control of the vehicle.

[0122] In summary, existing technologies for controlling vehicles using cockpit voice assistants are less efficient and comprehensive.

[0123] Based on the aforementioned technical problems, the technical concept of this invention is as follows: The purpose of controlling a vehicle is primarily to improve the user's driving experience, and users' control needs for the vehicle are similar or identical in the same scenario. With the increasing integration of sensors, cameras, microphones, and other hardware into vehicles, the vehicle's scenario can be determined based on this hardware. Therefore, a control command generation model can be pre-trained. This model can determine multiple other target control commands associated with the initial control command in the current scenario, based on the current scenario and the initial control command input by the user. Before training the control command generation model, a training set can be generated using the powerful common-sense reasoning capabilities of a large model. In this way, the user only needs to simply state their needs (initial control command), and the vehicle can execute a series of actions (target control commands) according to the current scenario to meet the user's needs. During this process, some functions or devices that the user is unfamiliar with or rarely uses will also be invoked, effectively improving the efficiency and comprehensiveness of controlling the vehicle using the cockpit voice assistant.

[0124] The technical solution of the present invention will now be described in detail through specific embodiments.

[0125] It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0126] Figure 1 This is a schematic flowchart of Embodiment 1 of the model training method provided by the present invention. Figure 1 As shown, the model training method may include the following steps:

[0127] S11. Obtain the first control command, the sample driving information of the sample vehicle, and the sample user information of the driver / rider of the sample vehicle.

[0128] It is understood that the executing entity of this embodiment of the invention can be a terminal device, such as a computer, tablet computer, or vehicle terminal, or a server, such as a backend processing platform. Therefore, this embodiment uses the term "electronic device" to refer to both terminal devices and servers for explanation and description. Whether the electronic device is specifically a terminal device or a server can be determined based on the actual situation.

[0129] It should be understood that the first control command, sample driving information, and sample user information can be obtained from historical driving data in a real driving environment, generated in a simulated driving environment, or generated based on Monte Carlo simulation. The method of obtaining the first control command, sample driving information, and sample user information can be specifically limited according to the actual situation, and the embodiments of the present invention do not impose specific limitations in this regard.

[0130] The first control command is used to control the first sample device in the sample vehicle.

[0131] For example, the first control command can be a voice command initiated by the user, such as "open the car window", "turn on the air conditioner", "turn on the ambiance", etc.

[0132] For example, the first sample device can be an air conditioner, a fragrance system, a car window, a car door, etc.

[0133] Optionally, the sample driving information includes at least one of the following: navigation data, weather data, and energy data.

[0134] For example, navigation data may include at least one of the following: the total mileage of the sample vehicle in this trip, the mileage driven continuously, the time driven continuously, and the remaining mileage; weather data may include at least one of the following: the interior temperature, the exterior temperature, the weather type, and the UV intensity; energy data may include the mileage that the sample vehicle can support with its remaining energy.

[0135] Optionally, the navigation data may also include the time the sample vehicle was closed inside.

[0136] Optionally, the sample user information includes at least one of the following: the number of users driving or riding in the sample vehicle, the age, gender, facial expression, actions, and location of each user in the sample vehicle, and the location of the speaking user in the sample vehicle.

[0137] It should be understood that the user who initiates the first control command can be the driver of the sample vehicle, or the front passenger, left rear passenger, right rear passenger, etc.

[0138] S12. Generate a prompt message based on the sample driving information, sample user information, and the first control command.

[0139] It should be understood that the prompt is used to guide the large model to output the desired result.

[0140] In one possible implementation, prompts can be generated based on sample driving information, sample user information, the first control command, and content related to the first control command in the knowledge base. If a knowledge base is pre-deployed in the large model, prompts can also be generated directly based on sample driving information, sample user information, and the first control command.

[0141] It should be understood that the knowledge base is used to explain the configuration options that can be used to configure various devices in different scenarios, as well as the advantages, disadvantages, and precautions of each configuration option.

[0142] For example, the knowledge base is explained below using Table 1.

[0143] Table 1

[0144]

[0145]

[0146] Referring to Table 1, the controllable configuration items and the contents of the remarks constitute the knowledge base.

[0147] Next, we will explain the generated prompts using two specific examples, taking the method of generating prompts based on sample driving information, sample user information, the first control command, and content related to the first control command in the knowledge base.

[0148] Example 1: Air conditioner prompts

[0149] Speaker information is as follows: Heart rate level: Medium, Heart rate variability level: Medium, Respiratory rate level: Medium, Blood oxygen level: Medium, Blood pressure level: Normal, Speaker's seat: Driver's seat, Appears to be: Middle-aged, Appears to be: Calm, Speaker's actions: No dangerous actions, Speaker's gender: Female, Car interior information is as follows: Number of people in the car: 1, Remaining mileage: Normal, Car interior temperature: Comfortable temperature, Weather data is as follows: Outside temperature: Comfortable temperature, Weather: Sunny, UV index: Medium, Duration in the car: A short period of time, Current approximate time: Noon.

[0150] Based on the preceding conditions, if the user does not specify the degree of device control or the level of detail in the settings, then the user can reasonably rewrite the second control command with multiple instructions. After rewriting, multiple devices and settings can be operated. For example, turning on the air conditioner could be rewritten as: turning on the air conditioner, setting the temperature to 22 degrees Celsius, adjusting the airflow to level 2, and directing the airflow towards the user's face. This makes the experience more personalized, catering to the needs of different genders and ages. It should also be tailored to the user's physical characteristics, allowing them to feel the cabin's comprehensive care.

[0151] The following information I have gathered regarding the operation of the air conditioner: 1. Mode: Cooling, Heating, Ventilation, Auto, Maximum Cooling 2. Temperature Adjustment: 17.5~32 degrees Celsius, increments of 0.5 degrees Celsius 3. Temperature Zone Synchronization: On / Off 4. Fan Speed: 0 to 3 levels 5. Airflow Direction: Facial, Foot, Facial / Foot, Defrosting 6. Air Purification: On / Off 7. Fresh Air System 8. Fragrance: Type, On / Off, Concentration 9. Air Conditioner Self-Drying: On / Off 10. Energy Saving Mode: On, Off, Energy Saving, Standard.

[0152] The user's first control command is: "Turn on the air conditioner." Rewrite five second control commands and output them in tabular form.

[0153] Example 2: Air conditioner prompts

[0154] Speaker information is as follows: Heart rate level: Medium, Heart rate variability level: Medium, Respiratory rate level: Medium, Blood oxygen level: Medium, Blood pressure level: Normal, Speaker's seat: Driver's seat, Appears to be: Middle-aged, Appears to be: Calm, Speaker's actions: No dangerous actions, Speaker's gender: Male. Vehicle information is as follows: Number of passengers: 1, Remaining mileage: Normal, Vehicle temperature: Comfortable temperature. Weather data is as follows: Outside temperature: Comfortable temperature, Weather: Sunny, UV index: High, Duration in the vehicle: Short time, Current approximate time: Morning.

[0155] Based on the preceding conditions, if the user does not specify the degree of device control or the level of detail in the settings, then the user can reasonably rewrite the first control command with multiple instructions. After rewriting, multiple devices and settings can be operated. For example, turning on the air conditioner could be rewritten as: turning on the air conditioner, setting the temperature to 22 degrees Celsius, adjusting the airflow to level 2, and directing the airflow towards the user's face. This makes the experience more personalized, catering to the needs of different genders and ages. It should also be tailored to the user's physical characteristics, allowing them to feel the cabin's comprehensive care.

[0156] I have collected the following information regarding the operable windows: 1. Designated position: Opening degree, Closing degree. 2. Car lock: Lock, Unlock. 3. Child lock: Lock, Unlock.

[0157] Background information, car window usage scenarios, and window opening suggestions:

[0158] Scenario 1: Someone is smoking inside the car.

[0159] - It is recommended to open the window on the same side as the smoker and lower the window to about one-third. This will allow the smoke to escape quickly without making the airflow inside the car too turbulent.

[0160] Scenario 2: It's raining but ventilation is needed.

[0161] - You can lower the driver's side window slightly, about 5 centimeters, and at the same time turn on the air conditioning to circulate the air inside the car to keep the air inside the car fresh.

[0162] Scenario 3: Traveling with pets

[0163] - If your pet is in the back seat, you can lower one of the back windows to about half its height to allow your pet to feel fresh air while preventing it from jumping out of the window.

[0164] Scenario 4: Nighttime driving

[0165] - Try to avoid opening the car windows too wide, as outside noise and light can affect your driving concentration. You can lower the front windows only a little bit to maintain a quiet and private interior.

[0166] Scene 5: Driving on a dusty road

[0167] - Close all windows and turn on the air conditioning to recirculate the air to prevent dust from entering the car.

[0168] Scenario 6: The vehicle is stationary but needs ventilation and heat dissipation.

[0169] - You can lower all four windows to about half their height to accelerate the heat exchange between the car interior and the outside.

[0170] Scene 7: Picking up and dropping off children at school

[0171] Before the child gets in the car, lower the rear windows for ventilation. After the child gets in, adjust the window height as needed based on the weather conditions.

[0172] Scene 8: Passing through an area with an unpleasant odor

[0173] - Quickly roll up all the windows and turn off the external air circulation to prevent odors from entering the car.

[0174] Scenario 9: On a long journey, passengers want to rest.

[0175] - You can close the front windows and only lower a small portion of the rear windows to reduce noise impact on rear passengers.

[0176] Scene 10: Driving on a mountain road

[0177] - Open the front windows appropriately to ensure ventilation inside the car, while avoiding unstable airflow that may affect driving due to the many curves on mountain roads.

[0178] Scene 11: Vehicle fully loaded

[0179] - Diagonal windows can be opened to enhance air circulation and reduce the stuffiness for passengers.

[0180] Scene 12: Passing through a noisy construction area

[0181] - Close all car windows to block out outside noise.

[0182] Scene 13: During periods of strong sunlight

[0183] - Lower the front sun visors and slightly open the rear windows to both ventilate the car and reduce direct sunlight.

[0184] Scene Fourteen: Driving in Winter

[0185] - Try not to open the car windows. If you need ventilation, you can open them briefly and then close them quickly.

[0186] Scene 15: Encountering a sandstorm

[0187] - Close the car windows completely to prevent sand and dust from entering the vehicle.

[0188] Scenario 16: The occupants of the vehicle feel stuffy, but the driver needs to concentrate.

[0189] - Only open the rear windows, and the opening should not be too large.

[0190] Scene 17: Traffic Congestion and Slow Movement

[0191] - Lower the front and rear windows halfway to improve air quality inside the car.

[0192] Scene 18: High humidity inside the car

[0193] - Lower all windows by one-third to speed up the removal of moisture.

[0194] Scene 19: The car interior was extremely hot when I first got in.

[0195] - First, open the car doors to ventilate, then lower all the windows while driving. After driving a certain distance, close the windows and turn on the air conditioning.

[0196] Scene 20: Driving on a narrow street

[0197] - Close the car windows to prevent tree branches and other objects from scratching them.

[0198] The user's first control command is: "Open the car window." Rewrite five second control commands and output them in tabular form.

[0199] S13. Input the prompt into the large model and obtain multiple second control commands output by the large model.

[0200] Among them, multiple second control commands are used to control the first sample device.

[0201] Optionally, multiple second control commands are also used to control a second sample device, which is a device in the sample vehicle that is associated with the first sample device in performing functions.

[0202] For example, assuming the first sample device is an air conditioner, the second sample device can be a fresh air system and a fragrance system. The air conditioner, fresh air system and fragrance system all have the function of air circulation and improving the air quality inside the vehicle.

[0203] In this approach, after subsequent model training based on the multiple second control commands, the trained control command generation model can generate multiple target control commands with similar functions to the initial control commands. This allows multiple devices to be invoked or multiple configuration items of the same device to be configured while meeting user needs. This solves the problem of some devices being idle due to users' lack of understanding of the full range of vehicle functions, and improves the comprehensiveness of vehicle control.

[0204] In one possible implementation, the prompt generated by S12 can also include a request for a response, such as: "After rewriting the second control command, please provide an explanatory response. This explanation is to be broadcast to the user, so please be concise, emotionally resonant, and acceptable to the user, making them feel cared for and giving them a first-class experience. The explanatory response should include various comfortable and respectful terms of address, and appropriately use some popular internet terms." In this way, the prompt can be input into the large model, and multiple second control commands and their corresponding responses can be obtained from the model's output.

[0205] In this approach, during subsequent model training based on these multiple second control commands, the trained control command generation model can output the corresponding target response. By playing the generated target response after executing the target control command generated by the control command generation model, the user's driving experience is effectively improved.

[0206] In another possible implementation, the prompt can be input into a large model to obtain multiple sample control instruction groups output by the large model. Each sample control instruction group includes multiple second control instructions and corresponding response statements.

[0207] In the above implementation, multiple sample control instruction groups are generated through a large model, so that multiple sample control instruction groups can be used as labels for model training, thereby improving the richness of target control instructions generated by the control instruction generation model during model inference.

[0208] S14. Based on sample driving information, sample user information, first control command and multiple second control commands, perform model training to obtain control command generation model.

[0209] Optionally, before training the model based on the second control instructions, a manual check can be performed to determine whether the generated second control instructions are reasonable. In other words, before model training, the second control instructions generated by the large model can be manually screened, and the remaining second control instructions after screening can be used for model training.

[0210] In one possible implementation, sample driving information, sample user information, and the first control command can be used as training samples, and multiple second control commands can be used as labels for the training samples. The initial model is trained using the training samples and the corresponding labels until the training cutoff condition is met, and the control commands obtained from the model training are used to generate the model.

[0211] Optionally, the initial model can be Extreme Gradient Boosting (XGBoost), or other types of models, such as neural network models, convolutional neural network models, decision tree models, etc., and the embodiments of the present invention do not impose specific limitations on this.

[0212] Optionally, the training cutoff condition is related to the initial model. Assuming the initial model is a neural network model, the training cutoff condition can be that the loss function value is less than a preset value. This embodiment of the invention does not impose specific restrictions on this.

[0213] In one possible implementation, a control command generation model is obtained by training the model based on sample driving information, sample user information, first control command, multiple second control commands, and response statements.

[0214] Specifically, sample driving information, sample user information, and the first control command can be used as training samples, and multiple second control commands and response statements can be used as labels for the training samples. The initial model can be trained using the training samples and corresponding labels until the training cutoff condition is met, and the control command generation model obtained from the model training can be obtained.

[0215] In this implementation, the response text is used as a label for the training sample to participate in the model training, so that the trained control command generation model has the ability to generate the target response text. When the target response text is played to users driving the target vehicle, it can effectively improve the user's driving experience.

[0216] In another possible implementation, command identifiers can be generated first based on multiple second control commands and their corresponding responses. Then, a model can be trained using sample driving information, sample user information, the first control commands, and the command identifiers to obtain a control command generation model.

[0217] Specifically, sample driving information, sample user information, and the first control command can be used as training samples, and the command identifier can be used as the label of the training sample. The initial model is trained using the training sample and the corresponding label until the training cutoff condition is met, and the control command generation model obtained from the model training is obtained.

[0218] In this implementation, by establishing a first mapping relationship among the response, control command, and command identifier, the command identifier can be used as a label for training samples to participate in model training. This effectively reduces the amount of data processing during model training and the amount of data output from the control command generation model during the inference phase, thereby improving model training efficiency and model inference efficiency. During model inference, based on the target command index generated by the control command generation model and the aforementioned first mapping relationship, the corresponding target control command and target response can be obtained.

[0219] In another possible implementation, a command identifier can be generated based on multiple sample control command groups. The model is then trained using sample driving information, sample user information, the first control command, and the command identifier to obtain a control command generation model.

[0220] Specifically, sample driving information, sample user information, and the first control command can be used as training samples, and the command identifier can be used as the label of the training sample. The initial model is trained using the training sample and the corresponding label until the training cutoff condition is met, and the control command generation model obtained from the model training is obtained.

[0221] For example, the second mapping relationship between sample control instruction groups and instruction identifiers can be represented by Table 2.

[0222] Table 2

[0223]

[0224] Referring to Table 2, the second control command includes specific temperature settings and airflow adjustments to meet the individual comfort needs of different users during driving. Meanwhile, the response is concise and clear, easily accepted by users, demonstrating care for their needs.

[0225] For example, the training samples and their corresponding labels can be represented by Table 3.

[0226] Table 3

[0227]

[0228] In this implementation, by establishing a second mapping relationship, instruction identifiers can be used as labels for training samples to participate in model training. This effectively reduces the amount of data processing during model training and the amount of data output from the control instruction generation model during the inference phase, thereby improving model training efficiency and model inference efficiency. During model inference, based on the target instruction indicators generated by the control instruction generation model and the aforementioned mapping relationship, the corresponding target control instructions and target responses can be obtained.

[0229] In this embodiment of the invention, a first control command, sample driving information of the sample vehicle, and sample user information of the driver / passenger in the sample vehicle are obtained. Then, a prompt is generated based on the sample driving information, sample user information, and the first control command. Next, the prompt is input into a large model to obtain multiple second control commands output by the large model. Finally, a control command generation model is obtained based on the sample driving information, sample user information, the first control command, and the multiple second control commands. The first control command is used to control a first sample device in the sample vehicle, and the multiple second control commands are used to control the first sample device. In this technical solution, multiple second control commands are generated based on the scene information of the sample vehicle (sample driving information and sample user information), the first control command, and the powerful common-sense reasoning ability of the large model. Furthermore, the scene information and the first control command are used as training samples, and the multiple second control commands are used as labels for model training to generate a control command generation model. This control command generation model has the ability to generate multiple target control commands associated with the functions corresponding to the initial control command, based on the initial control command and the current scene. In other words, by simply inputting an initial control command, a series of target control commands can be generated, enabling control of devices in the vehicle that the user is unaware of, thus improving the efficiency and comprehensiveness of vehicle control.

[0230] Figure 2 This is a schematic flowchart of Embodiment 2 of the model training method provided by the present invention. Figure 2 As shown, the model training method may include the following steps:

[0231] S21. Obtain the first control command.

[0232] S22. Obtain sample driving information and sample user information.

[0233] S23. Based on the first control command, sample driving information, and sample user information, generate a prompt and input it into the large model.

[0234] S24. Obtain multiple second control commands and response messages output by the large model.

[0235] S25. Input the first control command, sample driving information, sample user information, multiple second control commands and response statements into Xgboost for model training.

[0236] S26. After the training cutoff condition is met, obtain the control instructions to generate the model.

[0237] It should be understood that the steps in the embodiments of the present invention can be referred to Figure 1 The relevant details in the illustrated embodiments will not be repeated here.

[0238] After obtaining the control command generation model described above, it can be used to generate multiple target control commands, thereby controlling the vehicle. The following detailed description, in conjunction with specific embodiments, illustrates the method for controlling a vehicle using this control command generation model. These specific embodiments can be combined with each other; similar or identical concepts or processes may not be repeated in some embodiments.

[0239] In practice, the vehicle control method can be implemented by an electronic device with processing capabilities, such as a terminal or a server. It should be understood that the electronic device used for model training and the electronic device used for executing the vehicle control method can be the same device or different devices.

[0240] Figure 3 This is a schematic flowchart of a vehicle control method according to an embodiment of the present invention. Figure 3 As shown, the vehicle control method may include the following steps:

[0241] S31. After the target user driving the target vehicle initiates the initial control command, obtain the driving information of the target vehicle and the user information of the target vehicle.

[0242] The initial control command is used to control the first device in the target vehicle.

[0243] It should be understood that the relevant definition of target users can be found by referring to... Figure 1 The explanation of the speaking user in the illustrated embodiment, and the initial control commands can be referred to Figure 1The explanation of the first control command in the illustrated embodiment will not be repeated here.

[0244] The driving information includes at least one of the following: navigation data, weather data, and energy data.

[0245] User information includes at least one of the following: the number of users driving or riding in the target vehicle, the age, gender, facial expression, actions of each user, and their position in the target vehicle, and the position of the target user in the target vehicle.

[0246] Driving information can be referenced Figure 1 The explanation of the sample driving information in the illustrated embodiment, and the user information can be found by referring to... Figure 1 The explanation of the sample user information in the illustrated embodiment will not be repeated here.

[0247] In one possible implementation, the historical control commands generated by the control command generation model within a historical time period can be determined first. Then, after the target user initiates an initial control command, it is determined whether the initial control command and the historical control commands indicate control of the same configuration item on the same device. If the initial control command and the historical control commands do not indicate control of the same configuration item on the same device, then the driving information of the target vehicle and the user information of the driver / passenger of the target vehicle are obtained.

[0248] For example, the historical time periods are 5 seconds, 10 seconds, 15 seconds, etc.

[0249] In practical applications, suppose that at the first moment, the target user sends a voice command (i.e., the initial control command) "Turn on the air conditioner." The control command generation model generates multiple target control commands based on "Turn on the air conditioner," one of which is "Turn on the fresh air system." If at the second moment, the target user sends another voice command (i.e., the initial control command) "Turn off the fresh air system," and the first moment falls within a historical period of the second moment (e.g., within the first 5 seconds of the second moment), since the initial control command "Turn off the fresh air system" and the historical control command "Turn on the fresh air system" control the same configuration item (switch) of the same device (fresh air system), it indicates that the user does not like the target control command generated by the control command generation model. The user only wants to control a single device in the vehicle; therefore, only the initial control command is executed, thus controlling the vehicle.

[0250] If the initial control command and the historical control command do not indicate control of the same configuration item of the same device, it means that the user does not object to executing the target control command generated by the control command generation model. Then, the driving information of the target vehicle and the user information of the driver of the target vehicle are obtained to enable the subsequent model reasoning process.

[0251] In the above implementation method, user feedback is taken into account during the model inference process to avoid causing user resentment by executing the target control commands generated by the model, thus further taking into account the user experience and improving the accuracy of vehicle control.

[0252] S32. Input driving information, user information, and initial control commands into the control command generation model to obtain multiple target control commands.

[0253] The control command generation model is based on the above. Figure 1 and Figure 2 The model was trained using the model training method shown in the embodiment.

[0254] Among them, multiple target control commands are used to control the first device, and multiple target control commands are also used to control the second device, which is a device in the target vehicle that is associated with the first device to perform functions.

[0255] In this approach, multiple target control commands similar to the initial control execution function are generated based on the control command generation model. Multiple devices are invoked while meeting user needs, thereby solving the problem of some devices being idle due to users' lack of understanding of the full range of vehicle functions and improving the comprehensiveness of vehicle control.

[0256] In one possible implementation, the driving information, user information, and initial control commands can be input into a control command generation model to obtain a target command identifier. Then, multiple target control commands corresponding to the target command identifier, as well as the target response message corresponding to the target command identifier, can be obtained.

[0257] It should be understood that, referring to S13, a first mapping relationship or a second mapping relationship is predefined during the model training process. Based on the target instruction identifier and the first mapping relationship, multiple target control instructions and target response statements corresponding to the target instruction identifier can be determined; alternatively, based on the target instruction identifier and the second mapping relationship, multiple control instruction groups corresponding to the target instruction identifier can be determined, and then the target control instruction group can be determined from these multiple control instruction groups. The multiple control instructions contained in this control instruction group are the multiple target control instructions, and the response statements contained in this control instruction group are the target response statements.

[0258] Optionally, a control instruction group can be randomly selected as the target control instruction group, or the control instruction group with the largest number of controlled devices can be selected as the target control instruction group, or the control instruction group containing the most commonly used user settings can be selected as the target control instruction group.

[0259] For example, assuming that users frequently set the temperature and airflow of an air conditioner, then temperature and airflow are the user's frequently used device items.

[0260] In this implementation, the instruction identifier is used as the output of the control instruction generation model, which effectively reduces the amount of data processing during model training and the amount of data output of the control instruction generation model during the inference stage, thereby improving the model training efficiency and model inference efficiency.

[0261] Optionally, the number of instructions can be preset. If the number of target control instructions exceeds the preset number of instructions, the target control instructions generated by the control instruction generation model can be filtered so that the number of filtered target control instructions is less than or equal to the preset number of instructions.

[0262] S33. Execute multiple target control commands to control the target vehicle.

[0263] In one possible implementation, multiple target control commands can be modified based on pre-stored user preference parameters. These modified commands are then executed to control the target vehicle.

[0264] In practical applications, although the target control commands are derived by combining the initial control commands input by the target user and the current scenario of the target vehicle, the configuration parameters of multiple target control commands can be fine-tuned based on user preference data, taking into account the personalized needs of the target user.

[0265] For example, suppose the initial control command input by the target user is "turn on the air conditioner," and the target control command output by the control command generation model includes "adjust the air conditioner temperature to 25℃." Before executing this target control command, it is necessary to determine whether "25℃" is a user preference parameter in the current scenario. If "25℃" is a user preference parameter in the current scenario, or if there is no user preference parameter corresponding to the air conditioner temperature in the current scenario, then the target control command will not be modified. If the user preference parameter corresponding to the air conditioner temperature in the current scenario is not 25℃, then the user preference parameter will be assigned to the target control command to modify the target control command, and then the modified target control command will be executed to meet the user's personalized needs and further improve the user's driving experience.

[0266] For example, data points can be embedded in various devices within the vehicle. After each execution of a target control command, it can be determined whether the user resets the device configuration item corresponding to that target control command within a certain time period. For instance, suppose the target control command is "adjust the air conditioning temperature to 25℃". After executing the target control command, the user manually adjusts the air conditioning temperature to 24℃. Then, 24℃ is the user's preferred parameter for the air conditioning temperature configuration item in the current scenario.

[0267] Optionally, the target response can also be played to remind the user that the target control command (modified target control command) has been successfully executed, effectively improving the user experience.

[0268] This invention provides a vehicle control method. After a target user driving or riding in a target vehicle initiates an initial control command, the method acquires the vehicle's driving information and the user's information. Then, the driving information, user information, and the initial control command are input into a control command generation model to obtain multiple target control commands. Finally, the multiple target control commands are executed to control the target vehicle. Specifically, the initial control command is used to control a first device in the target vehicle, and the multiple target control commands are used to control the first device. The control command generation model is based on... Figure 1 and Figure 2 The model training method in the illustrated embodiment is used to train the model. In this technical solution, after the user inputs the initial control command, multiple target control commands that can be linked with the initial control command can be determined by combining the current scene of the target vehicle (driving information and user information). The user no longer needs to manually input multiple control commands; only one control command is needed to complete some vehicle control operations. This evolves from the existing "point-and-shoot" approach to a more comprehensive and collaborative approach, assisting the user in controlling in-vehicle devices and systems, improving the efficiency and comprehensiveness of vehicle control, and providing users with a more nuanced and attentive technological experience.

[0269] Figure 4 This is a schematic flowchart of a second embodiment of the vehicle control method provided by the present invention. Figure 4 As shown, the vehicle control method may include the following steps:

[0270] S41. Obtain voice commands.

[0271] It should be understood that voice commands are the initial control commands.

[0272] S42. Perform Natural Language Understanding (NLU) processing on the voice commands and obtain the NLU results.

[0273] S43. Convert the NLU result into a command (CMD).

[0274] It should be understood that CMD can be a prompt.

[0275] S44. Input CMD into the control command generation model.

[0276] S45. Determine if the control instruction generation model has recommended results. If it does, execute S46; otherwise, execute S49.

[0277] It should be understood that the recommended results are the target control commands and target response statements output by the control command generation model.

[0278] S46. Obtain multiple target control commands.

[0279] S47. Modify multiple target control instructions based on user preference parameters.

[0280] S48. Obtain the target response and the modified multiple target control instructions.

[0281] S49. To control the vehicle.

[0282] It should be understood that if there is no recommended result in S45, the vehicle will be controlled directly based on the user's voice command; if there is a recommended result in S45, the vehicle will be controlled based on the target response obtained in S48 and the modified multiple target control commands.

[0283] Figure 5 This is a schematic flowchart of a third embodiment of the vehicle control method provided by the present invention. Figure 5 As shown, the vehicle control method may include the following steps:

[0284] S51. Whether to enable the cross-modal command user interface (CUI) is recommended.

[0285] If enabled, proceed to step S52; otherwise, terminate.

[0286] The cross-modal CUI recommendation function can be enabled or disabled at the granularity of Vehicle Identification Number (VIN).

[0287] a. Enable: enginetype=tinnove-ai,domain=intelligent_car_control intent=cross_modal_cui_on Query: Enable cross-modal CUI recommendations.

[0288] b. Close: enginetype=tinnove-ai,domain=intelligent_car_control intent=cross_modal_cui_off.

[0289] c. Default: Off.

[0290] S52. Assemble driving information, user information, and initial control commands to generate a feature array.

[0291] The assembly process relies on real-time data services from the Internet of Things (IoT).

[0292] S53, Operation Control Command Generation Model.

[0293] Specifically, the generated feature array is input into the control command to generate the model.

[0294] S54. The control instruction generation model generates target response statements and control instructions for each target based on predefined recommendation results.

[0295] For example, the instruction generation model generates a target instruction identifier, determines multiple control instruction groups based on the target instruction identifier and the second mapping relationship, and determines the target control instruction group from the multiple control instruction groups, and determines the control instructions contained in the target control instruction group as the target control instruction.

[0296] It should be understood that the predefined recommendation result is the second mapping relationship.

[0297] In practical applications, this method can be fully implemented on the vehicle side.

[0298] Figure 6 This is a schematic flowchart of Embodiment 4 of the vehicle control method provided by the present invention. Figure 6 As shown, the vehicle control method may include the following steps:

[0299] S601, the voice assistant obtains the user's conversation.

[0300] S602, the voice assistant sends the conversation to the Automatic Speech Recognition (ASR) service.

[0301] S603, ASR service returns the processed text or NLU result.

[0302] S604, the voice assistant converts text or NLU results into CMD.

[0303] S605, the voice assistant sends CMD to the cross-modal CUI recommendation service.

[0304] S606, the cross-modal CUI recommendation service obtains driving information and user information through data collection services.

[0305] S607, the cross-modal CUI recommendation service generates a feature array based on CMD, driving information, and user information.

[0306] S608, Cross-modal CUI Recommendation Service Run Control Instruction Generation Model.

[0307] S609, Cross-modal CUI recommendation service obtains control instructions and generates multiple target control instructions from the model output.

[0308] S610, Cross-modal CUI recommendation service control instructions generate target response text for model output.

[0309] S611, the cross-modal CUI recommendation service sends the recommendation results to the voice assistant.

[0310] It should be understood that the recommended results consist of multiple target control instructions and target response statements.

[0311] S612, the voice assistant distributes tasks to the vehicle control application (APP) based on multiple target control commands and target response phrases.

[0312] S613 After the vehicle control APP executes multiple target control commands, it controls the Virtual Personal Assistant (VPA) service to display the control results on the screen.

[0313] S614, VPA service controls the text-to-speech (TTS) engine to read the target response.

[0314] It should be understood that the voice assistant, ASR service, vehicle control APP, VPA service, cross-modal CUI recommendation service, TTS engine, and data collection service are all deployed on the vehicle side.

[0315] In practical applications, this method can be deployed in the cloud and implemented through interaction between the vehicle and the cloud.

[0316] Figure 7 This is a flowchart illustrating Embodiment 5 of the vehicle control method provided by the present invention. Figure 7 As shown, the vehicle control method may include the following steps:

[0317] S701, the voice assistant obtains the user's conversation.

[0318] S702, the voice assistant sends the conversation to the cloud platform.

[0319] S703 and the cloud platform send the conversation to the ASR service.

[0320] S704 and ASR services return text or NLU results to the cloud platform.

[0321] S705 and the cloud platform send the text or NLU results to the NLU service for self-developed NLU processing.

[0322] S706, the NLU service returns the modified NLU result to the cloud platform.

[0323] S707, the cloud platform sends the cloud platform to the CMD service.

[0324] The S708 and CMD services convert the modified NLU results into CMD and send them to the cloud platform.

[0325] S709, Yunping sends CMD to the cross-modal CUI recommendation service (cloud).

[0326] The S710 and cross-modal CUI recommendation service (cloud) obtain driving and user information through vehicle status services.

[0327] S711, Cross-modal CUI Recommendation Service (Cloud) generates feature arrays based on CMD, driving information and user information.

[0328] S712, Cross-modal CUI Recommendation Service (Cloud) Run Control Instruction Generation Model.

[0329] S713, Cross-modal CUI Recommendation Service (Cloud) obtains control instructions and generates multiple target control instructions from the model output.

[0330] S714, Cross-modal CUI recommendation service (cloud) control instructions generate target response statements for model output.

[0331] S715, the cross-modal CUI recommendation service (cloud) sends the recommendation results to the cloud platform.

[0332] It should be understood that the recommended results consist of multiple target control instructions and target response statements.

[0333] The S716 and cloud platform send the recommendation results to the voice assistant.

[0334] The S717's voice assistant distributes tasks to the vehicle control app based on multiple target control commands and target response phrases.

[0335] After the S718 vehicle control APP executes multiple target control commands, it controls the VPA service to display the control results on the screen.

[0336] The S719 and VPA services control the TTS engine and broadcast the target response.

[0337] It should be understood that the voice assistant, vehicle control APP, VPA service, cross-modal CUI recommendation service (end), and TTS engine are all deployed on the vehicle side; while the cloud platform, NLU service, CMD service, cross-modal CUI recommendation service (cloud), vehicle status service, and ASR service are all deployed in the cloud.

[0338] In summary, this invention provides a more intelligent vehicle control method than existing technologies. It utilizes a large model to build a sufficient number of training samples, which are then manually reviewed before final model training. The resulting control command generation model responds very quickly and provides user-friendly feedback after executing the generated target control command. Furthermore, it can be deployed offline in the vehicle, protecting user privacy.

[0339] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the embodiments of the method of the present invention.

[0340] Figure 8 This is a schematic diagram of the structure of the model training device provided in an embodiment of the present invention. Figure 8 As shown, the model training device 80 includes:

[0341] The acquisition module 81 is used to acquire the first control command, the sample driving information of the sample vehicle, and the sample user information of the driver and passenger of the sample vehicle. The first control command is used to control the first sample device in the sample vehicle.

[0342] The generation module 82 is used to generate prompts based on the sample driving information, sample user information, and the first control command.

[0343] The input module 83 is used to input the prompt into the large model and obtain multiple second control commands output by the large model. The multiple second control commands are used to control the first sample device.

[0344] Training module 84 is used to train the model based on sample driving information, sample user information, first control command and multiple second control commands to obtain the control command generation model.

[0345] Furthermore, input module 83 is specifically used for:

[0346] Input the prompt into the large model to obtain multiple second control commands and corresponding responses output by the large model.

[0347] Correspondingly, training module 84 is specifically used for:

[0348] The model is trained based on sample driving information, sample user information, first control command, multiple second control commands, and response statements to obtain a control command generation model.

[0349] Furthermore, training module 84 is specifically used for:

[0350] Generate instruction identifiers based on multiple second control instructions and their corresponding responses.

[0351] The model is trained based on sample driving information, sample user information, first control command, and command identifier to obtain a control command generation model.

[0352] Furthermore, input module 83 is specifically used for:

[0353] Input the prompt into the large model to obtain multiple sample control instruction groups output by the large model. Each sample control instruction group includes multiple third control instructions and corresponding response messages.

[0354] Correspondingly, training module 84 is specifically used for:

[0355] The model is trained based on sample driving information, sample user information, the first control command, and multiple sample control command groups to obtain the control command generation model.

[0356] Furthermore, the sample driving information includes at least one of the following: navigation data, weather data, and energy data.

[0357] The sample user information includes at least one of the following: the number of users driving or riding in the sample vehicle, the age, gender, facial expression, actions, and location of each user in the sample vehicle, and the location of the speaking user in the sample vehicle.

[0358] Furthermore, multiple second control commands are also used to control a second sample device, which is a device in the sample vehicle that is associated with the first sample device in performing functions.

[0359] The model training apparatus provided in this embodiment of the invention can be used to execute the model training method in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0360] Figure 9 This is a schematic diagram of the vehicle control device provided in an embodiment of the present invention. Figure 9 As shown, the vehicle control device 90 includes:

[0361] The acquisition module 91 is used to acquire the driving information of the target vehicle and the information of the user driving the target vehicle after the target user initiates the initial control command. The initial control command is used to control the first device in the target vehicle.

[0362] The input module 92 is used to input driving information, user information, and initial control commands into the control command generation model to obtain multiple target control commands. These multiple target control commands are used to control the first device. The control command generation model is obtained by training the model based on the above model training method.

[0363] The execution module 93 is used to execute multiple target control commands to control the target vehicle.

[0364] Furthermore, input module 92 is specifically used for:

[0365] Input driving information, user information, and initial control commands into the control command generation model to obtain the target command identifier.

[0366] Retrieve multiple target control commands corresponding to the target command identifier.

[0367] Furthermore, after obtaining the target instruction identifier, the acquisition module 91 is also used for:

[0368] Obtain the target response message corresponding to the target instruction identifier.

[0369] Accordingly, after executing multiple target control commands to control the target vehicle, the vehicle control device 90 also includes:

[0370] The playback module is used to play the target reply.

[0371] Furthermore, execution module 93 is specifically used for:

[0372] Based on pre-stored user preference parameters, multiple target control commands are modified.

[0373] Execute the modified target control commands to control the target vehicle.

[0374] Furthermore, module 91 is specifically used for:

[0375] Determine the historical control commands generated by the control command generation model within the historical time period.

[0376] After the target user initiates the initial control command, determine whether the initial control command and the historical control command indicate that the same configuration item of the same device should be controlled.

[0377] If the initial control command and the historical control command do not indicate control of the same configuration item of the same device, then obtain the driving information of the target vehicle and the user information of the driver of the target vehicle.

[0378] Furthermore, driving information includes at least one of the following: navigation data, weather data, and energy data.

[0379] User information includes at least one of the following: the number of users driving or riding in the target vehicle, the age, gender, facial expression, actions of each user, and their position in the target vehicle, and the position of the target user in the target vehicle.

[0380] Furthermore, multiple target control commands are also used to control a second device, which is a device in the target vehicle that is associated with the first device in performing functions.

[0381] The vehicle control device provided in this embodiment of the invention can be used to execute the vehicle control method in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0382] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls, or entirely in hardware. Alternatively, some modules can be implemented through processing element calls in software, while others are implemented in hardware. Moreover, these modules can be fully or partially integrated together, or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions.

[0383] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 10 As shown, the electronic device 100 may include: a processor 101, a memory 102, and computer execution instructions stored in the memory 102 and executable on the processor 101. When the processor 101 executes the computer execution instructions, it implements the model training method or vehicle control method provided in any of the foregoing embodiments.

[0384] Optionally, the various devices of the electronic device 100 can be connected to each other via a system bus.

[0385] The memory 102 can be a separate memory unit or a memory unit integrated into the processor. The number of processors can be one or more.

[0386] Optionally, the electronic device 100 may also include a communication interface for interacting with other devices.

[0387] It should be understood that processor 101 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0388] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0389] All or part of the steps in the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above-described method embodiments. The aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0390] The electronic device provided in the embodiments of the present invention can be used to execute the model training method or vehicle control method provided in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0391] This invention provides a computer-readable storage medium storing computer-executable instructions. When these instructions are executed on a computer, the computer performs the aforementioned model training method or vehicle control method.

[0392] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0393] Optionally, a readable storage medium can be coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Alternatively, the readable storage medium can be an integral part of the processor. Both the processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components within the device.

[0394] This invention also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the above-described model training method or vehicle control method.

[0395] This invention also provides a vehicle including the aforementioned electronic device 100.

[0396] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A model training method, characterized in that, The method comprises: obtaining a first control instruction, sample driving information of a sample vehicle, and sample user information of a sample user riding the sample vehicle, the first control instruction being used for controlling a first sample device in the sample vehicle; generating a prompt according to the sample driving information, the sample user information, and the first control instruction; inputting the prompt into a large model to obtain a plurality of second control instructions output by the large model, the plurality of second control instructions being used for controlling the first sample device; performing model training based on the sample driving information, the sample user information, the first control instruction, and the plurality of second control instructions to obtain a control instruction generation model.

2. The method of claim 1, wherein, The method comprises: inputting the prompt into the large model to obtain the plurality of second control instructions output by the large model and corresponding reply sentences; correspondingly, the method comprises: performing model training based on the sample driving information, the sample user information, the first control instruction, the plurality of second control instructions, and the reply sentences to obtain the control instruction generation model.

3. The method of claim 2, wherein, The method comprises: generating instruction identifiers according to the plurality of second control instructions and the corresponding reply sentences; performing model training based on the sample driving information, the sample user information, the first control instruction, and the instruction identifiers to obtain the control instruction generation model.

4. The method according to claim 2 or 3, characterized in that, The method comprises: inputting the prompt into the large model to obtain a plurality of sample control instruction groups, each sample control instruction group comprising a plurality of second control instructions and corresponding reply sentences; correspondingly, the method comprises: performing model training based on the sample driving information, the sample user information, the first control instruction, and the plurality of sample control instruction groups to obtain the control instruction generation model.

5. The method according to any one of claims 1 to 3, characterized in that, The sample driving information comprises at least one of the following: navigation data, weather data, and energy data. The sample user information comprises at least one of the following: a number of users riding the sample vehicle, age, gender, expression, action, and position of each user in the sample vehicle, and position of the speaking user in the sample vehicle.

6. The method according to any one of claims 1 to 3, characterized in that, The plurality of second control instructions are also used for controlling a second sample device, the second sample device being a device in the sample vehicle that is associated with a function implemented by the first sample device.

7. A vehicle control method characterized by The method comprises: After a target user driving a target vehicle initiates an initial control instruction for controlling a first device in the target vehicle, driving information of the target vehicle and user information of the user driving the target vehicle are acquired; The driving information, the user information and the initial control instruction are input into a control instruction generation model to acquire a plurality of target control instructions for controlling the first device, the control instruction generation model being obtained by model training based on any one of claims 1-6; The plurality of target control instructions are executed to control the target vehicle.

8. The method of claim 7, wherein, The inputting of the driving information, the user information and the initial control instruction into the control instruction generation model to acquire the plurality of target control instructions comprises: The driving information, the user information and the initial control instruction are input into the control instruction generation model to acquire a target instruction identifier; The plurality of target control instructions corresponding to the target instruction identifier are acquired.

9. The method of claim 8, wherein, After the target instruction identifier is acquired, the method further comprises: A target reply corresponding to the target instruction identifier is acquired; Correspondingly, after the plurality of target control instructions are executed to control the target vehicle, the method further comprises: The target reply is played.

10. The method according to any one of claims 7-9, characterized in that, The execution of the plurality of target control instructions to control the target vehicle comprises: The plurality of target control instructions are modified according to pre-stored user preference parameters; The plurality of modified target control instructions are executed to control the target vehicle.

11. The method according to any one of claims 7-9, characterized in that, The acquisition of the driving information of the target vehicle and the user information of the user driving the target vehicle after the target user driving the target vehicle initiates the initial control instruction comprises: Historical control instructions generated by the control instruction generation model in a historical period are determined; After the target user initiates the initial control instruction, it is determined whether the initial control instruction and the historical control instructions indicate control of the same configuration item of the same device; If the initial control instruction and the historical control instructions do not indicate control of the same configuration item of the same device, the driving information of the target vehicle and the user information of the user driving the target vehicle are acquired.

12. The method according to any one of claims 7-9, characterized in that, The driving information comprises at least one of the following: navigation data, weather data, energy data; The user information comprises at least one of the following: the number of users driving the target vehicle, the age, gender, expression, action and position in the target vehicle of each user, and the position of the target user in the target vehicle.

13. The method according to any one of claims 7-9, characterized in that, The plurality of target control instructions are also used to control a second device, the second device being a device in the target vehicle associated with the first device to realize a function.

14. A model training apparatus, comprising: Comprise: An acquisition module is configured to acquire a first control instruction, sample driving information of a sample vehicle and sample user information of a user driving the sample vehicle, the first control instruction being used to control a first sample device in the sample vehicle; The generating module is configured to generate a prompt according to the sample driving information, the sample user information, and the first control instruction; The input module is configured to input the prompt into a large model to obtain a plurality of second control instructions output by the large model, the plurality of second control instructions being used for controlling the first sample device; The training module is configured to perform model training based on the sample driving information, the sample user information, the first control instruction, and the plurality of second control instructions to obtain a control instruction generation model.

15. A vehicle control device characterized by comprising: The method comprises: The obtaining module is configured to obtain driving information of a target vehicle and user information of a user driving the target vehicle after the user initiates an initial control instruction for controlling a first device in the target vehicle; The input module is configured to input the driving information, the user information, and the initial control instruction into a control instruction generation model to obtain a plurality of target control instructions for controlling the first device, the control instruction generation model being obtained by performing model training in the manner of any one of claims 1-6; The execution module is configured to execute the plurality of target control instructions to control the target vehicle.

16. An electronic device comprising: A processor, a memory, and computer-executable instructions stored on the memory and executable on the processor, wherein the processor, when executing the computer-executable instructions, is configured to implement the method of any one of claims 1-13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions, when executed by a processor, are configured to implement the method of any one of claims 1-13.

18. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, is configured to implement the method of any one of claims 1-13.

19. A vehicle characterized by comprising: The vehicle comprises the electronic device of claim 16.

Images