Vehicle operation control method and device, vehicle and storage medium

Abstract

The invention discloses a vehicle operation control method, device and equipment and a storage medium, and relates to the technical field of vehicles, and the method comprises the following steps: obtaining current environment data and a control behavior and a control instruction of a current user for a vehicle; inputting the current environment data, the control behavior and the control instruction into a pre-trained user habit prediction model, so that the user habit prediction model determines a prediction control instruction and control parameters of the current user for the vehicle according to the current environment data, the control behavior and the control instruction; determining an execution state of the prediction control instruction based on a feedback instruction of the current user for the prediction control instruction; and when the execution state is determined to be the determined execution state, the operation of the vehicle is controlled by executing the predictive control instruction based on the control parameters. According to the technical scheme, the vehicle-mounted voice assistant based on active execution can control the operation of the vehicle, the repeated instruction of a user is reduced, and the user experience is optimized.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of vehicle technology, and in particular, to a vehicle operation control method, device, vehicle, and storage medium. Background Art

[0002] As of now, the installation rate of intelligent voice interaction functions in Chinese passenger cars has reached as high as 86%. Car cockpits are developing towards intelligence, and in-vehicle voice assistants are an important core function in smart cockpits.

[0003] In the existing technology, in-vehicle voice assistants mainly rely on the passive method of user calls to complete user instructions or provide solutions.

[0004] Therefore, the present application proposes a vehicle operation control method, which controls vehicle operation based on an actively executed in-vehicle voice assistant. Summary of the Invention

[0005] The present invention provides a vehicle operation control method, device, vehicle and storage medium, which control vehicle operation based on an actively executed in-vehicle voice assistant, reduce the user's repeated instructions, and optimize the user experience.

[0006] In a first aspect, an embodiment of the present invention provides a method for controlling the operation of a vehicle, comprising:

[0007] Obtain current environmental data and the current user's control behavior and control instructions for the vehicle;

[0008] Inputting the current environment data, the control behavior, and the control instruction into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instruction and control parameters for the vehicle based on the current environment data, the control behavior, and the control instruction;

[0009] determining an execution state of the predictive control instruction based on a feedback instruction of the current user regarding the predictive control instruction;

[0010] When it is determined that the execution state is determined to be execution, the operation of the vehicle is controlled by executing the prediction control command based on the control parameter.

[0011] The technical solution of an embodiment of the present invention provides a vehicle operation control method, including: obtaining current environmental data and the current user's control behavior and control instructions for the vehicle; inputting the current environmental data, the control behavior and the control instructions into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data, the control behavior and the control instructions; determining the execution status of the predicted control instructions based on the current user's feedback instructions for the predicted control instructions; when it is determined that the execution status is confirmed to be executed, controlling the operation of the vehicle by executing the predicted control instructions based on the control parameters. The above technical solution can first obtain the current environmental data of the vehicle's current environment, and can also obtain the current user's control behavior and control instructions for the vehicle. Secondly, the current environmental data and the current user's control behavior and control instructions for the vehicle can be input into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data and the current user's control behavior and control instructions for the vehicle, and realize the prediction of the current user's next control instruction for the vehicle and the control parameters of the next control instruction. According to the current user's feedback instruction on the predicted control instruction, it is determined whether the predicted control instruction needs to be executed after the current control instruction, and the execution status of the predicted control instruction is determined to be confirmed execution or canceled execution according to the user's feedback instruction on the predicted control instruction. When the execution status is determined to be confirmed execution, an actively executed in-vehicle language assistant is run in the vehicle. The actively executed in-vehicle language assistant operates the corresponding device in the vehicle through the predicted control instructions and control parameters, and realizes the control of vehicle operation based on the actively executed in-vehicle voice assistant, reduces the user's repeated instructions, and optimizes the user experience.

[0012] Furthermore, after obtaining the current user's control behavior and control instructions for the vehicle, the following is also included:

[0013] Through comparison learning, the semantic information of the control behavior and the control instruction are aligned.

[0014] Furthermore, the process of constructing the to-be-trained model corresponding to the user habit prediction model includes:

[0015] Modeling the user's control behavior of the vehicle as a continuous behavior space, and constructing a Markov process based on the continuous behavior space;

[0016] Constructing a spatiotemporal feature map according to the execution time and execution position of each control behavior in the continuous behavior space;

[0017] The Markov process and the spatiotemporal feature map are combined to construct a model to be trained.

[0018] Furthermore, the training process of the user habit prediction model includes:

[0019] Acquire historical environment data, historical control behaviors and historical control instructions of each user on the vehicle under the historical environment, and the next control instruction corresponding to the historical control instruction, with the execution status being determined to be executed or canceled;

[0020] The historical environment data, the historical control behavior, the historical control instructions, and the next control instruction corresponding to the historical control instruction, whose execution status is to confirm execution and cancel execution, are used as training data to perform network training on the model to be trained until the model converges to obtain the user habit prediction model.

[0021] Furthermore, the historical environment data, the historical control behavior, the historical control instruction, and the next control instruction corresponding to the historical control instruction, the execution status of which is determined to be executed and canceled to be executed, are used as training data to perform network training on the to-be-trained model, including:

[0022] Determine the historical environment data, the historical control behavior, the historical control instruction, and the next control instruction whose execution status is determined to be executed as a positive sample, and determine the historical environment data, the historical control behavior, the historical control instruction, and the next control instruction whose execution status is canceled as a negative sample;

[0023] Network training is performed on the model to be trained based on the positive samples and the negative samples.

[0024] Furthermore, determining the execution state of the predictive control instruction based on the feedback instruction of the current user regarding the predictive control instruction includes:

[0025] When determining that the feedback instruction is positive feedback, determining the execution state to be confirmed execution;

[0026] When it is determined that the feedback instruction is negative feedback, the execution state is determined to be cancel execution.

[0027] Furthermore, it also includes:

[0028] When it is determined that the execution state is canceled, the vehicle is controlled to maintain the current state for operation.

[0029] In a second aspect, an embodiment of the present invention further provides a vehicle operation control device, comprising:

[0030] The acquisition module is used to obtain the current environment data and the current user's control behavior and control instructions for the vehicle;

[0031] a prediction module, configured to input the current environment data, the control behavior, and the control instruction into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instruction and control parameters for the vehicle based on the current environment data, the control behavior, and the control instruction;

[0032] a determination module, configured to determine an execution state of the predictive control instruction based on a feedback instruction of the current user regarding the predictive control instruction;

[0033] An execution module is used to control the operation of the vehicle by executing the prediction control instruction based on the control parameter when it is determined that the execution state is confirmed to be executed.

[0034] In a third aspect, an embodiment of the present invention further provides a vehicle, comprising:

[0035] at least one processor; and a memory communicatively coupled to the at least one processor;

[0036] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the vehicle operation control method as described in any one of the first aspects.

[0037] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to execute the vehicle operation control method as described in any one of the first aspects.

[0038] In a fifth aspect, the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed on a computer, the computer executes the vehicle operation control method provided in the first aspect.

[0039] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the vehicle's operation control device, or may be packaged separately from the processor of the vehicle's operation control device, and this application does not limit this.

[0040] The descriptions of the second, third, fourth and fifth aspects of this application can refer to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth and fifth aspects can refer to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0041] In this application, the name of the vehicle operation control device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalents.

[0042] These and other aspects of the present application will become more readily apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A flow chart of a vehicle operation control method provided by an embodiment of the present invention;

[0045] Figure 2 A flowchart of another vehicle operation control method provided by an embodiment of the present invention;

[0046] Figure 3 A schematic structural diagram of a vehicle operation control device provided by an embodiment of the present invention;

[0047] Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0049] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0050] The terms "first" and "second" and the like in the specification and drawings of this application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0051] Furthermore, the terms "including," "having," and any variations thereof, as used in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0052] It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although flow charts describe various operations (or steps) as sequential processes, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. When its operation is completed, the process can be terminated, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the features in the embodiments of the present invention and the embodiments can be combined with each other without conflict.

[0053] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being more preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0054] In the description of the present application, unless otherwise specified, “plurality” means two or more.

[0055] Figure 1 This is a flow chart of a vehicle operation control method provided by an embodiment of the present invention. This embodiment is applicable to the case where the vehicle operation control is achieved while reducing the frequency of user calls to the vehicle voice assistant. The method can be executed by the vehicle operation control device, such as Figure 1 As shown, the specific steps include:

[0056] Step 110: Acquire current environmental data and the current user's control behavior and control instructions for the vehicle.

[0057] Among them, the current environmental data is used to describe the current environment of the vehicle, which may specifically include temperature data and humidity data; the current user's control behavior of the vehicle can be understood as the user's control of the vehicle based on actual operations, for example, it may include adjusting the seat angle, raising and lowering windows, adjusting the rearview mirror angle, etc.; the current user's control instructions for the vehicle can be understood as the user's control of the vehicle through voice, for example, it may include "brighten the lights", "turn on the air conditioner", etc.

[0058] Specifically, vehicles are typically equipped with sensors. Therefore, current environmental data, specifically temperature and humidity data, can be acquired based on these sensors. Control of the vehicle based on actual operations can be implemented using the vehicle's controller. The vehicle's memory stores the control actions of each user prior to the current moment. Therefore, the current user's control actions and control instructions for the vehicle can be determined from the memory based on the current user's user ID, enabling rapid acquisition of the current user's control actions and control instructions for the vehicle.

[0059] In the embodiment of the present invention, the acquisition of the current environment data of the vehicle and the acquisition of the current user's control behavior and control instructions for the vehicle are achieved.

[0060] Step 120: Input the current environmental data, the control behavior, and the control instructions into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data, the control behavior, and the control instructions.

[0061] Among them, the user habit prediction model is obtained by training the Markov process constructed by the continuous behavior space obtained by modeling the user's control behavior of the vehicle based on historical environmental data and the historical control behaviors and historical control instructions of each user for the vehicle in the historical environment, as well as the next control instructions corresponding to the historical control instructions, and the to-be-trained model constructed by the spatiotemporal feature map constructed by the execution time and execution position of each control behavior in the continuous behavior space. The user habit prediction model can predict the current user's predicted control instructions for the vehicle and the control parameters corresponding to the predicted control instructions based on the environmental data of the vehicle's environment and the current user's control behaviors and control instructions for the vehicle.

[0062] Therefore, after obtaining the current environmental data and the current user's control behavior and control instructions for the vehicle, the current environmental data, control behavior and control instructions can be input as input data into a pre-trained user habit prediction model. The user habit prediction model can determine the current user's predicted control instructions for the vehicle and the control parameters corresponding to the predicted control instructions based on the current environmental data, control behavior and control instructions.

[0063] The control parameter corresponding to the predictive control instruction can be understood as the degree of control of the corresponding device by the predictive control instruction. For example, when the predictive control instruction is to turn on the air conditioner, the control parameter can be the air conditioner temperature.

[0064] In the embodiment of the present invention, the next control instruction of the current user to the vehicle is predicted, providing a data basis for the next operation control of the vehicle.

[0065] Step 130: Determine the execution status of the predictive control instruction based on the feedback instruction of the current user regarding the predictive control instruction.

[0066] Among them, the feedback instruction can be positive feedback or negative feedback. Positive feedback can be an affirmative response, such as "OK", "That's right", "OK", etc., and negative feedback can be a negative response, such as "Not needed", "Cancel", "NO", etc.

[0067] Specifically, after determining the current user's predicted control instructions and control parameters for the vehicle, the predicted control instructions can be broadcast. For example, the language broadcast is performed through a sound device. In the process of the sound device broadcasting the predicted control instructions, the current user can make feedback on the predicted control instructions through voice information, and then the feedback instructions can be determined based on the voice feedback made by the current user on the predicted control instructions, that is, the feedback instructions can be determined by extracting keywords from the feedback information. In the case of determining that the feedback instruction is an affirmative reply, the execution status of the predicted control instruction can be determined to be confirmed to be executed, that is, the predicted control instruction can be executed after the current control instruction is executed. In the case of determining that the feedback instruction is a negative reply, the execution status of the predicted control instruction can be determined to be canceled to be executed, that is, there is no need to execute the predicted control instruction after the current control instruction is executed.

[0068] In an embodiment of the present invention, whether a predictive control instruction needs to be executed after the current control instruction is determined based on the user's feedback on the predictive control instruction based on language feedback or trigger feedback, so that the execution status of the predictive control instruction is determined to be confirmed execution or canceled execution based on the user's feedback on the predictive control instruction.

[0069] Step 140: When it is determined that the execution state is confirmed to be executed, control the operation of the vehicle by executing the predictive control instruction based on the control parameter.

[0070] Specifically, when the execution state of the predictive control instruction is determined to be execution-determined, the predictive control instruction is determined to be executed after the current control instruction is executed. Therefore, the predictive control instruction can be executed after the current control instruction is executed. Specifically, the predictive control instruction can be executed based on a control parameter corresponding to the predictive control instruction, and the operation of a corresponding device in the vehicle can be controlled based on the control parameter. For example, when the predictive control instruction is to turn on the air conditioner and the control parameter is the air conditioner temperature, the vehicle air conditioner can be controlled to operate based on the air conditioner temperature.

[0071] In an embodiment of the present invention, when the execution state is determined to be confirmed execution, an actively executed in-vehicle language assistant is run in the vehicle. The actively executed in-vehicle language assistant realizes the control operation of the corresponding device in the vehicle by predicting control instructions and control parameters, thereby realizing the control of vehicle operation based on the actively executed in-vehicle voice assistant.

[0072] The vehicle operation control method provided by an embodiment of the present invention includes: obtaining current environmental data and the current user's control behavior and control instructions for the vehicle; inputting the current environmental data, the control behavior and the control instructions into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data, the control behavior and the control instructions; determining the execution status of the predicted control instructions based on the current user's feedback instructions for the predicted control instructions; when it is determined that the execution status is confirmed to be executed, controlling the operation of the vehicle by executing the predicted control instructions based on the control parameters. The above technical solution can first obtain the current environmental data of the vehicle's current environment, and can also obtain the current user's control behavior and control instructions for the vehicle. Secondly, the current environmental data and the current user's control behavior and control instructions for the vehicle can be input into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data and the current user's control behavior and control instructions for the vehicle, and realize the prediction of the current user's next control instruction for the vehicle and the control parameters of the next control instruction. According to the current user's feedback instruction on the predicted control instruction, it is determined whether the predicted control instruction needs to be executed after the current control instruction, and the execution status of the predicted control instruction is determined to be confirmed execution or canceled execution according to the user's feedback instruction on the predicted control instruction. When the execution status is determined to be confirmed execution, an actively executed in-vehicle language assistant is run in the vehicle. The actively executed in-vehicle language assistant operates the corresponding device in the vehicle through the predicted control instructions and control parameters, and realizes the control of vehicle operation based on the actively executed in-vehicle voice assistant, reduces the user's repeated instructions, and optimizes the user experience.

[0073] Figure 2 This is a flow chart of another vehicle operation control method provided by an embodiment of the present invention. This embodiment is specific based on the above embodiment. Figure 2 As shown, in this embodiment, the method may further include:

[0074] Step 210: Acquire current environmental data and the current user's control behavior and control instructions for the vehicle.

[0075] Specifically, current environmental data can be obtained based on the sensors installed in the vehicle, specifically temperature data and humidity data. The current user's control behavior and control instructions for the vehicle can be understood as the current user's control behavior and control instructions for the vehicle in a historical time period. The vehicle's memory stores the control behavior of each user for the vehicle before the current moment. For example, the control behavior of each user for the vehicle can be stored based on the user information and time information of each user. Therefore, the current user's control behavior and control instructions for the vehicle in the historical time period can be determined in the memory based on the current user's voiceprint information or facial information and the time identifier of the historical time period, thereby realizing the rapid acquisition of the current user's control behavior and control instructions for the vehicle.

[0076] In the embodiment of the present invention, the acquisition of the current environment data of the vehicle and the acquisition of the current user's control behavior and control instructions for the vehicle are achieved.

[0077] Step 220: Align the semantic information of the control behavior and the control instruction through comparison and learning.

[0078] Among them, control behavior can be understood as the user's operating behavior data on various vehicle devices, such as adjusting seats, window control, etc., and control instructions can be understood as voice instructions issued by users to control various vehicle devices, such as lowering seats, opening windows, etc.

[0079] Specifically, for the same control methods across various devices within the vehicle, the semantic information of the corresponding control actions and control instructions can be aligned through comparison learning. This means aligning the descriptive information of the corresponding control actions and control instructions. For example, "adjust seat" and "lower seat" can be aligned to "adjust seat."

[0080] In the embodiment of the present invention, by aligning the semantic information of the control behavior and the control instruction, a unified description of the control method of each device in the vehicle is achieved in the control behavior dimension and the control instruction dimension.

[0081] Step 230: Input the current environmental data, the control behavior and the control instructions into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data, the control behavior and the control instructions.

[0082] In one embodiment, the process of constructing the to-be-trained model corresponding to the user habit prediction model includes:

[0083] The user's control behavior of the vehicle is modeled as a continuous behavior space, and a Markov process is constructed based on the continuous behavior space; a spatiotemporal feature map is constructed according to the execution time and execution location of each control behavior in the continuous behavior space; and a model to be trained is constructed by combining the Markov process and the spatiotemporal feature map.

[0084] In one embodiment, the training process of the user habit prediction model includes:

[0085] Obtain historical environment data and historical control behaviors and historical control instructions of each user for the vehicle under the historical environment, as well as the next control instruction corresponding to the historical control instruction; determine the historical environment data, the historical control behaviors, the historical control instructions, and the next control instruction corresponding to the historical control instruction with an execution status of confirmed execution as positive samples, and determine the historical environment data, the historical control behaviors, the historical control instructions, and the next control instruction corresponding to the historical control instruction with an execution status of canceled execution as negative samples; perform network training on the model to be trained based on the positive samples and the negative samples until the model converges, thereby obtaining the user habit prediction model.

[0086] Specifically, the user's control behavior of the vehicle includes seat angle, window lifting, rearview mirror angle, etc. First, the user's control behavior of the vehicle can be modeled as a continuous action space, and a Markov process can be constructed based on the continuous behavior space. Secondly, the vehicle's driving trajectory, the user's driving habits (such as throttle opening, seat angle, steering wheel angle, etc.), and the user's control instructions for the vehicle can be combined to determine the execution time and execution position corresponding to each control behavior in the continuous action space. Then, a spatiotemporal feature map can be constructed based on the execution time and execution position corresponding to each control behavior, and then a model to be trained can be constructed based on the Markov process and the spatiotemporal feature map.

[0087] Furthermore, multi-objective optimization can be achieved by designing a reward function for the model to be trained. The optimization objectives can include seat angle, perceived temperature, lighting suitability, window status, rearview mirror field of view, and personalized needs. The value of the reward function can comprehensively reflect the performance of these optimization objectives after the state transition, allowing the model to automatically and appropriately balance various optimization objectives while maximizing a single cumulative reward, thereby finding the optimal next control instruction.

[0088] It's important to note that a hierarchical optimization approach can be used to prioritize each optimization objective. For example, priority 1 is safety, priority 2 is personalized needs, priority 3 is comfort and health, and priority 4 is energy efficiency. Priorities are implicitly reflected in the reward function through weighting, with higher-priority objectives receiving the highest weights.

[0089] Furthermore, the model to be trained can be trained. Specifically, first, the historical environmental data of vehicle operation and the historical control behaviors and historical control instructions of each user for the vehicle in the historical environment and the next control instruction corresponding to the historical control instruction can be obtained; the historical environmental data, historical control behaviors and historical control instructions and the next control instruction corresponding to the historical control instruction with the execution status of confirmed execution are determined as positive samples, and the historical environmental data, historical control behaviors and historical control instructions and the next control instruction corresponding to the historical control instruction with the execution status of canceled execution are determined as negative samples; network training is performed on the model to be trained based on the positive samples and negative samples until the model converges to obtain a user habit prediction model.

[0090] In practical applications, model training can be performed locally in the vehicle. The training method can adopt weakly supervised multi-task learning. Model training is completed through a multi-task learning architecture. The loss function is designed and the incremental learning mechanism is integrated to complete the fine-tuning of the model.

[0091] In addition, based on the labeled driving interaction logs (including the current user's feedback instructions for the predicted control instructions and the next control instructions), with multi-classification accuracy as the main goal, combined with confidence calibration and user feedback signals for optimization, continuous learning and model updates can be carried out.

[0092] In the embodiment of the present invention, the next control instruction of the current user to the vehicle is predicted, providing a data basis for the next operation control of the vehicle.

[0093] Step 240: Determine the execution status of the predictive control instruction based on the feedback instruction of the current user regarding the predictive control instruction.

[0094] In one implementation, step 240 may specifically include:

[0095] When it is determined that the feedback instruction is positive feedback, the execution state is determined to be confirmed execution; when it is determined that the feedback instruction is negative feedback, the execution state is determined to be canceled execution.

[0096] Specifically, after determining the current user's prediction control instructions and control parameters for the vehicle, the prediction control instructions can be broadcasted by voice through the sound device, and the prediction control instructions can also be displayed in text through the display device. In the process of the display device displaying the prediction control instructions based on text, the current user can make feedback on the prediction control instructions through trigger information, and then the feedback instruction can be determined based on the trigger feedback made by the current user on the prediction control instructions. For example, the feedback instruction can be determined to be "OK" based on "Confirm to execute according to the broadcast instruction", that is, the feedback instruction can be determined as an affirmative reply. In the case of determining that the feedback instruction is an affirmative reply, the execution status of the prediction control instruction can be determined to be confirmed to be executed. The feedback instruction can be determined to be "Cancel" based on the user triggering the interactive button corresponding to "No" in the interactive device, that is, the feedback instruction can be determined to be a negative reply. In the case of determining that the feedback instruction is a negative reply, the execution status of the prediction control instruction can be determined to be canceled to be executed.

[0097] In an embodiment of the present invention, whether a predictive control instruction needs to be executed after the current control instruction is determined based on the user's feedback on the predictive control instruction based on language feedback or trigger feedback, so that the execution status of the predictive control instruction is determined to be confirmed execution or canceled execution based on the user's feedback on the predictive control instruction.

[0098] Step 250: When it is determined that the execution state is confirmed to be executed, control the operation of the vehicle by executing the predictive control instruction based on the control parameter.

[0099] Specifically, when it is determined that the execution status of the predictive control instruction is determined to be executed, it is determined that the predictive control instruction can be executed after executing the current control instruction, that is, the operation of the vehicle can be controlled by executing the predictive control instruction. Specifically, the predictive control instruction can be executed based on the control parameters to control the operation of the corresponding device in the vehicle.

[0100] Step 260: When it is determined that the execution state is canceled, control the vehicle to maintain the current state for operation.

[0101] Specifically, when it is determined that the execution state of the prediction control instruction is to cancel execution, it is determined that there is no need to execute the prediction control instruction after executing the current control instruction, that is, the vehicle can be controlled to maintain the current state for operation.

[0102] In an embodiment of the present invention, when it is determined that the execution state is confirmed to be executed, an actively executed in-vehicle language assistant is run in the vehicle. The actively executed in-vehicle language assistant realizes the control operation of the corresponding device in the vehicle by predicting the control instructions and control parameters. When it is determined that the execution state is canceled, the vehicle is controlled to maintain the current state for operation, thereby realizing the control of the vehicle operation based on the actively executed in-vehicle voice assistant.

[0103] The vehicle operation control method provided by an embodiment of the present invention includes: obtaining current environmental data and the current user's control behavior and control instructions for the vehicle; aligning the semantic information of the control behavior and the control instructions through comparative learning; inputting the current environmental data, the control behavior and the control instructions into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data and the control behavior and the control instructions; determining the execution status of the predicted control instructions based on the current user's feedback instructions for the predicted control instructions; when it is determined that the execution status is confirmed execution, controlling the operation of the vehicle by executing the predicted control instructions based on the control parameters; when it is determined that the execution status is canceled execution, controlling the vehicle to maintain the current state for operation. The above technical solution can first obtain the current environmental data of the vehicle's current environment, and can also obtain the current user's control behavior and control instructions for the vehicle. Secondly, it can align the semantic information of the control behavior and control instructions through comparative learning, and realize a unified description of the control methods of various devices in the vehicle in the control behavior dimension and the control instruction dimension. Then, the current environmental data and the current user's control behavior and control instructions for the vehicle can be input into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data and the current user's control behavior and control instructions for the vehicle, and realize the next control instruction of the current user for the vehicle and the next control parameter. Predict the control parameters of the control instruction, determine whether the predicted control instruction needs to be executed after the current control instruction based on the current user's feedback instruction on the predicted control instruction, and determine the execution status of the predicted control instruction as confirmed execution or canceled execution based on the user's feedback instruction on the predicted control instruction. When the execution status is determined to be confirmed execution, run the actively executed in-vehicle language assistant in the vehicle. The actively executed in-vehicle language assistant realizes the control operation of the corresponding device in the vehicle through the predicted control instruction and control parameters. When the execution status is determined to be canceled execution, control the vehicle to maintain the current state for operation, and realize the control of vehicle operation based on the actively executed in-vehicle voice assistant, reduce the user's repeated instructions, and optimize the user experience.

[0104] Figure 3This is a schematic diagram of the structure of a vehicle operation control device provided by an embodiment of the present invention. This device can be used to reduce the frequency of user calls to an in-vehicle voice assistant while also achieving vehicle operation control. This device can be implemented using software and / or hardware and is typically integrated into an electronic device, such as a vehicle.

[0105] like Figure 3 As shown, the device includes:

[0106] An acquisition module 310 is used to acquire current environmental data and the current user's control behavior and control instructions for the vehicle;

[0107] Prediction module 320, configured to input the current environment data, the control behavior, and the control instruction into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instruction and control parameters for the vehicle based on the current environment data, the control behavior, and the control instruction;

[0108] a determination module 330, configured to determine an execution state of the predictive control instruction based on a feedback instruction of the current user regarding the predictive control instruction;

[0109] The execution module 340 is configured to control the operation of the vehicle by executing the prediction control instruction based on the control parameter when determining that the execution state is determined to be execution.

[0110] The vehicle operation control device provided in this embodiment obtains current environmental data and the current user's control behavior and control instructions for the vehicle; inputs the current environmental data, the control behavior and the control instructions into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data, the control behavior and the control instructions; determines the execution status of the predicted control instructions based on the current user's feedback instructions for the predicted control instructions; when it is determined that the execution status is confirmed to be executed, controls the operation of the vehicle by executing the predicted control instructions based on the control parameters. The above technical solution can first obtain the current environmental data of the vehicle's current environment, and can also obtain the current user's control behavior and control instructions for the vehicle. Secondly, the current environmental data and the current user's control behavior and control instructions for the vehicle can be input into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control instructions and control parameters for the vehicle based on the current environmental data and the current user's control behavior and control instructions for the vehicle, and realize the prediction of the current user's next control instruction for the vehicle and the control parameters of the next control instruction. According to the current user's feedback instruction on the predicted control instruction, it is determined whether the predicted control instruction needs to be executed after the current control instruction, and the execution status of the predicted control instruction is determined to be confirmed execution or canceled execution according to the user's feedback instruction on the predicted control instruction. When the execution status is determined to be confirmed execution, an actively executed in-vehicle language assistant is run in the vehicle. The actively executed in-vehicle language assistant operates the corresponding device in the vehicle through the predicted control instructions and control parameters, and realizes the control of vehicle operation based on the actively executed in-vehicle voice assistant, reduces the user's repeated instructions, and optimizes the user experience.

[0111] Based on the above embodiment, the device further includes:

[0112] The comparison module is used to align the semantic information of the control behavior and the control instruction through comparison learning.

[0113] In one embodiment, the process of constructing the to-be-trained model corresponding to the user habit prediction model includes:

[0114] The user's control behavior of the vehicle is modeled as a continuous behavior space, and a Markov process is constructed based on the continuous behavior space; a spatiotemporal feature map is constructed according to the execution time and execution location of each control behavior in the continuous behavior space; and a model to be trained is constructed by combining the Markov process and the spatiotemporal feature map.

[0115] In one embodiment, the training process of the user habit prediction model includes:

[0116] Obtain historical environment data and historical control behaviors and historical control instructions of each user on the vehicle under the historical environment, as well as the next control instruction with the execution status of confirming execution and canceling execution corresponding to the historical control instructions; use the historical environment data, the historical control behaviors, the historical control instructions, and the next control instruction with the execution status of confirming execution and canceling execution corresponding to the historical control instructions as training data to perform network training on the model to be trained until the model converges, thereby obtaining the user habit prediction model.

[0117] Furthermore, the historical environment data, the historical control behavior, the historical control instruction, and the next control instruction corresponding to the historical control instruction, the execution status of which is determined to be executed and canceled to be executed, are used as training data to perform network training on the to-be-trained model, including:

[0118] The historical environment data, the historical control behavior, the historical control instruction, and the execution status corresponding to the historical control instruction is the next control instruction to be determined to be executed as positive samples, and the historical environment data, the historical control behavior, the historical control instruction, and the execution status corresponding to the historical control instruction is the next control instruction to be canceled to be determined as negative samples; network training is performed on the model to be trained based on the positive samples and the negative samples.

[0119] Based on the above embodiment, the determination module 330 is specifically configured to:

[0120] When it is determined that the feedback instruction is positive feedback, the execution state is determined to be confirmed execution; when it is determined that the feedback instruction is negative feedback, the execution state is determined to be canceled execution.

[0121] Based on the above embodiment, the execution module 340 is further configured to:

[0122] When it is determined that the execution state is canceled, the vehicle is controlled to maintain the current state for operation.

[0123] The vehicle operation control device provided in the embodiment of the present invention can execute the vehicle operation control method provided in any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the vehicle operation control method.

[0124] It is worth noting that in the embodiment of the above-mentioned vehicle operation control device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0125] Figure 4 A schematic structural diagram of a vehicle provided in an embodiment of the present invention. Figure 4 A block diagram of an exemplary vehicle 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The vehicle 4 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0126] like Figure 4 As shown, vehicle 4 is represented as a general purpose computing electronic device. Components of vehicle 4 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0127] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0128] The vehicle 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the vehicle 4, including volatile and non-volatile media, removable and non-removable media.

[0129] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The vehicle 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be configured to read and write to non-removable, non-volatile magnetic media ( Figure 4 Not shown, often called a "hard drive"). Although Figure 4 Not shown, a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0130] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0131] The vehicle 4 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the vehicle 4, and / or any device that enables the vehicle 4 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the vehicle 4 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. Figure 4 As shown, the network adapter 20 communicates with other modules of the vehicle 4 via the bus 18. Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the vehicle 4, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0132] The processing unit 16 executes various functional applications and page displays by running programs stored in the system memory 28, such as implementing the vehicle operation control method provided by an embodiment of the present invention, which includes:

[0133] Obtain current environmental data and the current user's control behavior and control instructions for the vehicle;

[0134] Inputting the current environment data, the control behavior, and the control instruction into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instruction and control parameters for the vehicle based on the current environment data, the control behavior, and the control instruction;

[0135] determining an execution state of the predictive control instruction based on a feedback instruction of the current user regarding the predictive control instruction;

[0136] When it is determined that the execution state is determined to be execution, the operation of the vehicle is controlled by executing the prediction control command based on the control parameter.

[0137] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the vehicle operation control method provided by any embodiment of the present invention.

[0138] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, for example, a vehicle operation control method provided by an embodiment of the present invention is implemented. The method includes:

[0139] Obtain current environmental data and the current user's control behavior and control instructions for the vehicle;

[0140] Inputting the current environment data, the control behavior, and the control instruction into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instruction and control parameters for the vehicle based on the current environment data, the control behavior, and the control instruction;

[0141] determining an execution state of the predictive control instruction based on a feedback instruction of the current user regarding the predictive control instruction;

[0142] When it is determined that the execution state is determined to be execution, the operation of the vehicle is controlled by executing the prediction control command based on the control parameter.

[0143] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0144] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0145] Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0146] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0147] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computing device. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they can be implemented using program code executable by a computer device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

[0148] In addition, the acquisition, storage, use, and processing of data in the technical solution of the present invention comply with relevant provisions of laws and regulations.

[0149] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A vehicle operation control method, characterized in that: include: Obtain current environmental data and the current user's control behavior and control instructions for the vehicle; Inputting the current environment data, the control behavior, and the control instruction into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instruction and control parameters for the vehicle based on the current environment data, the control behavior, and the control instruction; determining an execution state of the predictive control instruction based on a feedback instruction of the current user regarding the predictive control instruction; When it is determined that the execution state is determined to be execution, the operation of the vehicle is controlled by executing the prediction control command based on the control parameter.

2. The vehicle operation control method according to claim 1, characterized in that: After obtaining the current user's control behavior and control instructions for the vehicle, it also includes: Through comparison learning, the semantic information of the control behavior and the control instruction are aligned.

3. The vehicle operation control method according to claim 1, characterized in that: The process of constructing the to-be-trained model corresponding to the user habit prediction model includes: Modeling the user's control behavior of the vehicle as a continuous behavior space, and constructing a Markov process based on the continuous behavior space; Constructing a spatiotemporal feature map according to the execution time and execution position of each control behavior in the continuous behavior space; The Markov process and the spatiotemporal feature map are combined to construct a model to be trained.

4. The vehicle operation control method according to claim 3, characterized in that: The training process of the user habit prediction model includes: Acquire historical environment data, historical control behaviors and historical control instructions of each user on the vehicle under the historical environment, and the next control instruction corresponding to the historical control instruction, with the execution status being determined to be executed or canceled; The historical environment data, the historical control behavior, the historical control instructions, and the next control instruction corresponding to the historical control instruction, whose execution status is to confirm execution and cancel execution, are used as training data to perform network training on the model to be trained until the model converges to obtain the user habit prediction model.

5. The vehicle operation control method according to claim 4, characterized in that: The historical environment data, the historical control behavior, the historical control instruction, and the next control instruction corresponding to the historical control instruction, the execution status of which is determined to be executed or canceled, are used as training data to perform network training on the to-be-trained model, including: Determine the historical environment data, the historical control behavior, the historical control instruction, and the next control instruction whose execution status is determined to be executed as a positive sample, and determine the historical environment data, the historical control behavior, the historical control instruction, and the next control instruction whose execution status is canceled as a negative sample; Network training is performed on the model to be trained based on the positive samples and the negative samples.

6. The vehicle operation control method according to claim 1, characterized in that: Determining an execution state of the predictive control instruction based on a feedback instruction of the current user regarding the predictive control instruction includes: When determining that the feedback instruction is positive feedback, determining the execution state to be confirmed execution; When it is determined that the feedback instruction is negative feedback, the execution state is determined to be cancel execution.

7. The vehicle operation control method according to claim 1, characterized in that: Also includes: When it is determined that the execution state is canceled, the vehicle is controlled to maintain the current state for operation.

8. A vehicle operation control device, characterized in that: include: The acquisition module is used to obtain the current environment data and the current user's control behavior and control instructions for the vehicle; a prediction module, configured to input the current environment data, the control behavior, and the control instruction into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control instruction and control parameters for the vehicle based on the current environment data, the control behavior, and the control instruction; a determination module, configured to determine an execution state of the predictive control instruction based on a feedback instruction of the current user regarding the predictive control instruction; An execution module is used to control the operation of the vehicle by executing the prediction control instruction based on the control parameter when it is determined that the execution state is confirmed to be executed.

9. A vehicle, characterized in that: The vehicle comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the vehicle operation control method as described in any one of claims 1 to 7.

10. A storage medium containing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the vehicle operation control method according to any one of claims 1 to 7.

Images