A vehicle operation control method and device, vehicle, and storage medium

Abstract

The application discloses a vehicle operation control method, device and equipment and a storage medium, relates to the technical field of vehicles, and comprises the following steps: acquiring current environment data and a control behavior and a control instruction of a current user for a vehicle; inputting the current environment data, the control behavior and the control instruction into a pre-trained user habit prediction model, so that the user habit prediction model determines a predicted control instruction and a control parameter of the current user for the vehicle according to the current environment data and the control behavior and the control instruction; determining an execution state of the predicted control instruction based on a feedback instruction of the current user for the predicted control instruction; and when it is determined that the execution state is determined execution, controlling the operation of the vehicle by executing the predicted control instruction based on the control parameter. The above technical scheme realizes that the vehicle operation is controlled by the vehicle-mounted voice assistant based on active execution, reduces the repeated instruction of the user, and optimizes the user experience.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and more particularly to a vehicle operation control method, device, vehicle, and storage medium. Background Technology

[0002] Currently, the adoption rate of intelligent voice interaction functions in passenger cars in China has reached 86%, and the car cockpit is developing towards intelligence. In-vehicle voice assistants are an important core function in intelligent cockpits.

[0003] In existing technologies, in-vehicle voice assistants mainly rely on a passive approach, where users call out commands to complete user instructions or provide solutions.

[0004] Therefore, this application proposes a vehicle operation control method based on an actively executed in-vehicle voice assistant to control vehicle operation. Summary of the Invention

[0005] This invention provides a vehicle operation control method, device, vehicle, and storage medium to control vehicle operation based on an actively executed in-vehicle voice assistant, reducing the need for users to repeatedly issue commands and optimizing the user experience.

[0006] In a first aspect, embodiments of the present invention provide a vehicle operation control method, comprising:

[0007] Acquire current environmental data, as well as the current user's control behavior and commands regarding the vehicle;

[0008] The current environmental data, the control behavior, and the control command are input into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control command and control parameters for the vehicle based on the current environmental data, the control behavior, and the control command.

[0009] The execution status of the predictive control command is determined based on the current user's feedback instructions to the predictive control command;

[0010] When the execution state is determined to be "determined execution", the vehicle's operation is controlled by executing the predictive control command based on the control parameters.

[0011] The technical solution of this invention provides a vehicle operation control method, comprising: acquiring current environmental data and current user control behavior and control commands for the vehicle; inputting the current environmental data, the control behavior, and the control commands into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control commands and control parameters for the vehicle based on the current environmental data, the control behavior, and the control commands; determining the execution state of the predicted control commands based on the current user's feedback commands to the predicted control commands; and when the execution state is determined to be "determined execution," controlling the vehicle's operation by executing the predicted control commands based on the control parameters. The above technical solution first acquires the current environmental data of the vehicle's current environment, as well as the current user's control behavior and commands. Second, it inputs this environmental data and the user's control behavior and commands into a pre-trained user habit prediction model. This model then determines the predicted control commands and parameters for the user's next control command and its parameters. Based on the user's feedback on the predicted control command, it determines whether the command should be executed after the current command, thus determining whether the execution status of the predicted control command is "confirmed execution" or "cancelled execution." If the execution status is "confirmed execution," an actively executed in-vehicle voice assistant runs in the vehicle. This actively executed in-vehicle voice assistant operates corresponding devices in the vehicle using the predicted control commands and parameters, enabling vehicle control based on an actively executed in-vehicle voice assistant. This reduces the need for users to repeatedly issue commands and optimizes the user experience.

[0012] Furthermore, after acquiring the current user's control behavior and control commands for the vehicle, it also includes:

[0013] By comparing and learning, the semantic information of the control behavior and the control instructions is aligned.

[0014] Furthermore, the process of constructing the training model corresponding to the user habit prediction model includes:

[0015] The user's control behavior on the vehicle is modeled as a continuous behavior space, and a Markov process is constructed based on the continuous behavior space.

[0016] A spatiotemporal feature map is constructed based on the execution time and execution location of each control behavior in the continuous behavior space;

[0017] The model to be trained is constructed by combining the Markov process and the spatiotemporal feature map.

[0018] Furthermore, the training process of the user habit prediction model includes:

[0019] The acquisition of historical environmental data, as well as the historical control behaviors and commands of each user for the vehicle under the historical environment, and the execution status of the corresponding historical control commands, determines the next control command to be executed or canceled.

[0020] The historical environment data, historical control behaviors, historical control instructions, and the next control instructions whose execution states are determined to be executed or canceled are used as training data to train the network of the model to be trained until the model converges, thereby obtaining the user habit prediction model.

[0021] Further, the historical environment data, the historical control behaviors, the historical control instructions, and the next control instructions whose execution states are determined to be executed or canceled are used as training data to train the network of the model to be trained, including:

[0022] The historical environmental data, the historical control behavior, the historical control instructions, and the next control instruction whose execution status is determined to be executed are identified as positive samples; the historical environmental data, the historical control behavior, the historical control instructions, and the next control instruction whose execution status is cancelled are identified as negative samples.

[0023] The network is trained based on the positive and negative samples.

[0024] Further, determining the execution state of the predictive control command based on the current user's feedback instructions to the predictive control command includes:

[0025] When the feedback instruction is determined to be positive feedback, the execution state is determined to be execution.

[0026] When the feedback instruction is determined to be negative feedback, the execution status is determined to be canceled.

[0027] Furthermore, it also includes:

[0028] When the execution status is determined to be canceled, the vehicle is controlled to maintain its current state of operation.

[0029] Secondly, embodiments of the present invention also provide a vehicle operation control device, comprising:

[0030] The acquisition module is used to acquire current environmental data as well as the current user's control behavior and control commands for the vehicle.

[0031] The prediction module is used to input the current environmental data, the control behavior, and the control command into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control command and control parameters for the vehicle based on the current environmental data, the control behavior, and the control command.

[0032] The determination module is used to determine the execution status of the predictive control instruction based on the current user's feedback instructions to the predictive control instruction;

[0033] An execution module is used to control the operation of the vehicle by executing the predictive control command based on the control parameters when the execution state is determined to be determined execution.

[0034] Thirdly, embodiments of the present invention also provide a vehicle, the vehicle comprising:

[0035] At least one processor; and a memory communicatively connected to said at least one processor;

[0036] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the vehicle operation control method as described in any of the first aspects.

[0037] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, characterized in that the computer-executable instructions, when executed by a computer processor, are used to perform the vehicle operation control method as described in any of the first aspects.

[0038] Fifthly, this application provides a computer program product including computer instructions that, when executed on a computer, cause the computer to perform the vehicle operation control method as provided in the first aspect.

[0039] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the vehicle's operation control device, or it may be packaged separately from the processor of the vehicle's operation control device; this application does not impose any limitations on this.

[0040] The descriptions of the second, third, fourth, and fifth aspects in this application can be referred to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referred to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0041] In this application, the names of the aforementioned vehicle operation control devices do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those in this application, they fall within the scope of the claims of this application and their equivalents.

[0042] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart illustrating a vehicle operation control method provided in an embodiment of the present invention;

[0045] Figure 2 A flowchart of another vehicle operation control method provided in an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the structure of a vehicle operation control device provided in an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0048] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0049] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0050] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0051] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0052] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc. Moreover, embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0053] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0054] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0055] Figure 1 This is a flowchart illustrating a vehicle operation control method provided in an embodiment of the present invention. This embodiment is applicable to situations where vehicle operation control is achieved while reducing the frequency of user calls to the in-vehicle voice assistant. The method can be executed by a vehicle operation control device, such as... Figure 1 As shown, the specific steps include the following:

[0056] Step 110: Obtain current environmental data and current user control behavior and commands for the vehicle.

[0057] The current environmental data describes the current environment of the vehicle, which may include temperature and humidity data. The current user's control behavior on the vehicle can be understood as the user's control of the vehicle based on actual operation, such as adjusting the seat angle, raising and lowering the windows, and adjusting the rearview mirror angle. The current user's control commands on the vehicle can be understood as the user's control of the vehicle through voice, such as "turn on the lights" or "turn on the air conditioning".

[0058] Specifically, vehicles are typically equipped with sensors, allowing the acquisition of current environmental data, such as temperature and humidity. Vehicle control can be implemented using the vehicle's controller. The vehicle's memory stores the control actions of each user up to the current moment. Therefore, the current user's control actions and commands can be quickly retrieved from the memory based on the user's identifier.

[0059] In this embodiment of the invention, the acquisition of the current environmental data of the vehicle and the acquisition of the current user's control behavior and control commands on the vehicle are realized.

[0060] Step 120: Input the current environment data, the control behavior, and the control command into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control command and control parameters for the vehicle based on the current environment data, the control behavior, and the control command.

[0061] The user habit prediction model is obtained by training a Markov process constructed from the continuous behavior space obtained by modeling the user's control behavior of the vehicle, and a spatiotemporal feature map constructed from the execution time and execution location of each control behavior in the continuous behavior space, based on historical environmental data, historical control behaviors and commands of each user on the vehicle in the historical environment, and the next control command corresponding to the historical control command. The user habit prediction model can predict the current user's predicted control command for the vehicle and the corresponding control parameters based on the environmental data of the vehicle's environment and the current user's control behavior and commands for the vehicle.

[0062] Therefore, after obtaining the current environmental data and the current user's control behavior and control commands for the vehicle, the current environmental data, control behavior, and control commands can be used as input data into the pre-trained user habit prediction model. The user habit prediction model can determine the current user's predicted control commands for the vehicle and the corresponding control parameters based on the current environmental data, control behavior, and control commands.

[0063] The control parameters corresponding to predictive control commands can be understood as the degree of control the predictive control command exerts on the corresponding device. For example, when the predictive control command is to turn on the air conditioner, the control parameter can be the air conditioner temperature.

[0064] In this embodiment of the invention, the next control command from the current user to the vehicle is predicted, providing a data basis for the next step of vehicle operation control.

[0065] Step 130: Determine the execution status of the predictive control instruction based on the current user's feedback instructions to the predictive control instruction.

[0066] The feedback instructions can be positive or negative. Positive feedback can be affirmative responses, such as "OK", "That's right", etc., while negative feedback can be negative responses, such as "Not needed", "Cancel", "NO" etc.

[0067] Specifically, after determining the current user's predictive control command and control parameters for the vehicle, the predictive control command can be broadcast. For example, it can be broadcast via a voice device. During the broadcast of the predictive control command, the current user can provide feedback on the predictive control command through voice information. The feedback command can then be determined based on the user's voice feedback, i.e., by extracting keywords from the feedback information. If the feedback command is determined to be a positive response, the execution status of the predictive control command can be set to "determined execution," meaning that the predictive control command can be executed after the current control command is executed. If the feedback command is determined to be a negative response, the execution status of the predictive control command can be set to "cancelled execution," meaning that the predictive control command does not need to be executed after the current control command is executed.

[0068] In this embodiment of the invention, the system determines whether a predictive control instruction needs to be executed after the current control instruction based on the user's feedback on the predictive control instruction, either by language feedback or trigger feedback. This allows the system to determine the execution status of the predictive control instruction as either "determined to execute" or "cancelled to execute" based on the user's feedback.

[0069] Step 140: When the execution state is determined to be "determined execution", the vehicle operation is controlled by executing the predictive control command based on the control parameters.

[0070] Specifically, when the execution status of a predictive control command is determined to be "determined to execute," it is determined that the predictive control command will be executed after the current control command. Therefore, the predictive control command can be executed after the current control command. More specifically, the predictive control command can be executed based on the control parameters corresponding to it, and the operation of the corresponding device in the vehicle can be controlled based on these control parameters. For example, if the predictive control command is to turn on the air conditioner and the control parameter is the air conditioner temperature, the vehicle's air conditioner can be controlled to operate based on the air conditioner temperature.

[0071] In this embodiment of the invention, when the execution state is determined to be executed, an actively executed in-vehicle voice assistant is run in the vehicle. The actively executed in-vehicle voice assistant controls the operation of the corresponding device in the vehicle by predicting control commands and control parameters, thereby realizing vehicle operation control based on the actively executed in-vehicle voice assistant.

[0072] The vehicle operation control method provided in this embodiment of the invention includes: acquiring current environmental data and current user control behavior and control commands for the vehicle; inputting the current environmental data, the control behavior, and the control commands into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control commands and control parameters for the vehicle based on the current environmental data, the control behavior, and the control commands; determining the execution state of the predicted control commands based on the current user's feedback commands to the predicted control commands; and controlling the vehicle's operation by executing the predicted control commands based on the control parameters when the execution state is determined to be "determined execution". The above technical solution first acquires the current environmental data of the vehicle's current environment, as well as the current user's control behavior and commands. Second, it inputs this environmental data and the user's control behavior and commands into a pre-trained user habit prediction model. This model then determines the predicted control commands and parameters for the user's next control command and its parameters. Based on the user's feedback on the predicted control command, it determines whether the command should be executed after the current command, thus determining whether the execution status of the predicted control command is "confirmed execution" or "cancelled execution." If the execution status is "confirmed execution," an actively executed in-vehicle voice assistant runs in the vehicle. This actively executed in-vehicle voice assistant operates corresponding devices in the vehicle using the predicted control commands and parameters, enabling vehicle control based on an actively executed in-vehicle voice assistant. This reduces the need for users to repeatedly issue commands and optimizes the user experience.

[0073] Figure 2 This is a flowchart illustrating another vehicle operation control method provided in an embodiment of the present invention. This embodiment is a specific modification based on the above embodiments. Figure 2 As shown, in this embodiment, the method may further include:

[0074] Step 210: Obtain current environmental data and current user control behavior and commands for the vehicle.

[0075] Specifically, current environmental data, such as temperature and humidity data, can be acquired based on the sensors installed in the vehicle. The current user's control behaviors and commands towards the vehicle can be understood as the user's control behaviors and commands towards the vehicle within a historical time period. The vehicle's memory stores the control behaviors of each user before the current moment. For example, the control behaviors of each user can be stored based on their user information and time information. Therefore, based on the current user's voiceprint or facial information and the time stamp of the historical time period, the current user's control behaviors and commands towards the vehicle within the historical time period can be determined from the memory, enabling rapid acquisition of the current user's control behaviors and commands.

[0076] In this embodiment of the invention, the acquisition of the current environmental data of the vehicle and the acquisition of the current user's control behavior and control commands on the vehicle are realized.

[0077] Step 220: Align the semantic information of the control behavior and the control instructions through comparative learning.

[0078] Among them, control behavior can be understood as the user's operational behavior data on various devices in the vehicle, such as adjusting the seat and controlling the windows, while control commands can be understood as the voice commands issued by the user to control various devices in the vehicle, such as lowering the seat and opening the window.

[0079] Specifically, for the same control methods across all devices within a vehicle, the semantic information of the corresponding control behaviors and control commands can be aligned through comparative learning, that is, the descriptive information of the corresponding control behaviors and control commands can be aligned. For example, "adjust seat" and "lower seat" can be aligned to "adjust seat".

[0080] In this embodiment of the invention, by aligning the semantic information of control behaviors and control commands, a unified description of the control methods of various devices in the vehicle is achieved in both the control behavior dimension and the control command dimension.

[0081] Step 230: Input the current environment data, the control behavior, and the control command into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control command and control parameters for the vehicle based on the current environment data, the control behavior, and the control command.

[0082] In one implementation, the process of constructing the training model corresponding to the user habit prediction model includes:

[0083] The user's control behavior on the vehicle is modeled as a continuous behavior space, and a Markov process is constructed based on the continuous behavior space. A spatiotemporal feature map is constructed according to the execution time and execution location of each control behavior in the continuous behavior space. The Markov process and the spatiotemporal feature map are combined to construct a model to be trained.

[0084] In one implementation, the training process of the user habit prediction model includes:

[0085] Acquire historical environmental data, as well as the historical control behaviors and commands of each user for the vehicle under the historical environment, and the next control command corresponding to the historical control command; determine the historical environmental data, the historical control behaviors, the historical control commands, and the next control command whose execution state is determined to be executed as positive samples, and determine the historical environmental data, the historical control behaviors, the historical control commands, and the next control command whose execution state is cancelled as negative samples; train the network model to be trained based on the positive samples and the negative samples until the model converges to obtain the user habit prediction model.

[0086] Specifically, user control behaviors of the vehicle include seat angle, window raising and lowering, rearview mirror angle, etc. First, the user's control behaviors of the vehicle can be modeled as a continuous action space, and a Markov process can be constructed based on the continuous action space. Second, the execution time and execution location of each control behavior in the continuous action space can be determined by combining the vehicle's driving trajectory, the user's driving habits (such as throttle opening, seat angle, steering wheel angle, etc.), and the user's control commands to the vehicle. Then, a spatiotemporal feature map can be constructed based on the execution time and execution location of each control behavior. Finally, a model to be trained can be constructed based on the Markov process and the spatiotemporal feature map.

[0087] Furthermore, multi-objective optimization can be achieved by designing a reward function for the model to be trained. Optimization objectives can include seat angle, perceived temperature, lighting suitability, window status, rearview mirror field of view, and personalized needs. The value of the reward function can comprehensively reflect the performance of the above optimization objectives after state transitions, enabling the model to automatically and reasonably balance various optimization objectives while maximizing a single cumulative reward, and find the optimal next control instruction.

[0088] It should be noted that a hierarchical optimization method can be used to clearly define the priority order of each optimization objective. For example: priority 1 is safety, priority 2 is personalized needs, priority 3 is comfort and health, and priority 4 is energy efficiency. The priority is implicitly reflected in the reward function by designing weights, with higher-priority objectives having significantly higher weights.

[0089] Then, the model to be trained can be trained. Specifically, firstly, historical environmental data of vehicle operation, as well as historical control behaviors and commands of each user and the next control command corresponding to the historical command can be obtained. The historical environmental data, historical control behaviors and commands, and the next control command corresponding to the historical command with the execution status of "determined to execute" are identified as positive samples, and the historical environmental data, historical control behaviors and commands, and the next control command corresponding to the historical command with the execution status of "cancelled to execute" are identified as negative samples. The network model to be trained is trained based on the positive and negative samples until the model converges, thus obtaining the user habit prediction model.

[0090] In practical applications, model training can be performed locally on the vehicle. The training method can adopt weakly supervised multi-task learning, complete the model training through a multi-task learning architecture, design a loss function and integrate an incremental learning mechanism to complete the fine-tuning of the model.

[0091] Furthermore, based on labeled driving interaction logs (including the current user's feedback instructions to the predictive control command and the next control command), with multi-classification accuracy as the primary goal, it can be optimized by combining confidence calibration and user feedback signals to continuously learn and update the model.

[0092] In this embodiment of the invention, the next control command from the current user to the vehicle is predicted, providing a data basis for the next step of vehicle operation control.

[0093] Step 240: Determine the execution status of the predictive control instruction based on the current user's feedback instructions to the predictive control instruction.

[0094] In one implementation, step 240 may specifically include:

[0095] When the feedback instruction is determined to be positive feedback, the execution state is determined to be "determined to execute"; when the feedback instruction is determined to be negative feedback, the execution state is determined to be "cancelled to execute".

[0096] Specifically, after determining the current user's predictive control commands and control parameters for the vehicle, the predictive control commands can be broadcast verbally via a sound device or displayed as text on a display device. During the text-based display of the predictive control commands, the current user can provide feedback to the commands via trigger information. The feedback command can then be determined based on the user's trigger feedback. For example, if the user confirms "execute according to the broadcast command," the feedback command is determined to be "confirm," thus confirming a positive response. If the feedback command is confirmed as positive, the execution status of the predictive control command is determined to be "confirmed execution." Alternatively, if the user triggers the "no" button on the interactive device, the feedback command is determined to be "cancel," thus confirming a negative response. If the feedback command is confirmed as negative, the execution status of the predictive control command is determined to be "cancel execution."

[0097] In this embodiment of the invention, the system determines whether a predictive control instruction needs to be executed after the current control instruction based on the user's feedback on the predictive control instruction, either by language feedback or trigger feedback. This allows the system to determine the execution status of the predictive control instruction as either "determined to execute" or "cancelled to execute" based on the user's feedback.

[0098] Step 250: When the execution state is determined to be "determined execution", the vehicle operation is controlled by executing the predictive control command based on the control parameters.

[0099] Specifically, when the execution status of the predictive control command is determined to be "determined to execute", it is determined that the predictive control command can be executed after the current control command is executed. That is, the operation of the vehicle can be controlled by executing the predictive control command. In particular, the predictive control command can be executed based on the control parameters to control the operation of the corresponding device in the vehicle.

[0100] Step 260: When the execution status is determined to be canceled, control the vehicle to maintain the current state and continue running.

[0101] Specifically, when the execution status of the predictive control command is determined to be canceled, it is determined that no predictive control command needs to be executed after the current control command is executed, and the vehicle can be controlled to maintain its current state.

[0102] In this embodiment of the invention, when the execution state is determined to be "determined execution", an actively executed in-vehicle voice assistant is run in the vehicle. The actively executed in-vehicle voice assistant controls the operation of the corresponding device in the vehicle by predicting control commands and control parameters. When the execution state is determined to be "cancelled execution", the vehicle is controlled to maintain its current state and run, thereby realizing vehicle operation control based on the actively executed in-vehicle voice assistant.

[0103] The vehicle operation control method provided in this embodiment of the invention includes: acquiring current environmental data and current user control behavior and control commands for the vehicle; aligning the semantic information of the control behavior and the control commands through comparative learning; inputting the current environmental data, the control behavior, and the control commands into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control commands and control parameters for the vehicle based on the current environmental data, the control behavior, and the control commands; determining the execution state of the predicted control commands based on the current user's feedback commands to the predicted control commands; when the execution state is determined to be "determined execution," controlling the vehicle's operation by executing the predicted control commands based on the control parameters; when the execution state is determined to be "cancelled execution," controlling the vehicle to maintain its current state of operation. The above technical solution firstly acquires current environmental data of the vehicle's current environment, as well as the user's current control behaviors and commands. Secondly, through comparative learning, it aligns the semantic information of control behaviors and commands, achieving a unified description of the control methods of various devices within the vehicle across both the control behavior and command dimensions. Furthermore, the current environmental data and the user's current control behaviors and commands are input into a pre-trained user habit prediction model. This model then determines the predicted control commands and parameters for the user's next control command and the next... The system predicts control parameters for control commands and determines whether the predicted control command should be executed after the current command based on the user's feedback. This allows for determining whether the predicted control command should be executed (either confirmed or cancelled) based on user feedback. If the execution status is confirmed, an in-vehicle voice assistant is activated, controlling the corresponding devices in the vehicle using the predicted control command and parameters. If the execution status is cancelled, the vehicle maintains its current state. This proactive in-vehicle voice assistant control reduces the need for repeated commands and optimizes the user experience.

[0104] Figure 3This is a schematic diagram of a vehicle operation control device according to an embodiment of the present invention. This device is applicable to situations where vehicle operation control can be achieved while reducing the frequency of user calls to the in-vehicle voice assistant. The device can be implemented through software and / or hardware and is generally integrated into electronic devices, such as vehicles.

[0105] like Figure 3 As shown, the device includes:

[0106] The acquisition module 310 is used to acquire current environmental data and the current user's control behavior and control commands for the vehicle.

[0107] The prediction module 320 is used to input the current environmental data, the control behavior, and the control command into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control command and control parameters for the vehicle based on the current environmental data, the control behavior, and the control command.

[0108] The determination module 330 is used to determine the execution status of the predictive control instruction based on the current user's feedback instruction to the predictive control instruction;

[0109] The execution module 340 is used to control the operation of the vehicle by executing the predictive control command based on the control parameters when the execution state is determined to be determined execution.

[0110] The vehicle operation control device provided in this embodiment acquires current environmental data and the current user's control behavior and control commands for the vehicle; inputs the current environmental data, the control behavior, and the control commands into a pre-trained user habit prediction model, so that the user habit prediction model determines the current user's predicted control commands and control parameters for the vehicle based on the current environmental data, the control behavior, and the control commands; determines the execution state of the predicted control commands based on the current user's feedback commands to the predicted control commands; when the execution state is determined to be "determined execution", the device controls the operation of the vehicle by executing the predicted control commands based on the control parameters. The above technical solution first acquires the current environmental data of the vehicle's current environment, as well as the current user's control behavior and commands. Second, it inputs this environmental data and the user's control behavior and commands into a pre-trained user habit prediction model. This model then determines the predicted control commands and parameters for the user's next control command and its parameters. Based on the user's feedback on the predicted control command, it determines whether the command should be executed after the current command, thus determining whether the execution status of the predicted control command is "confirmed execution" or "cancelled execution." If the execution status is "confirmed execution," an actively executed in-vehicle voice assistant runs in the vehicle. This actively executed in-vehicle voice assistant operates corresponding devices in the vehicle using the predicted control commands and parameters, enabling vehicle control based on an actively executed in-vehicle voice assistant. This reduces the need for users to repeatedly issue commands and optimizes the user experience.

[0111] Based on the above embodiments, the device further includes:

[0112] The comparison module is used to align the semantic information of the control behavior and the control instructions through comparison learning.

[0113] In one implementation, the process of constructing the training model corresponding to the user habit prediction model includes:

[0114] The user's control behavior on the vehicle is modeled as a continuous behavior space, and a Markov process is constructed based on the continuous behavior space. A spatiotemporal feature map is constructed according to the execution time and execution location of each control behavior in the continuous behavior space. The Markov process and the spatiotemporal feature map are combined to construct a model to be trained.

[0115] In one implementation, the training process of the user habit prediction model includes:

[0116] Historical environmental data, as well as the historical control behaviors and commands of each user for the vehicle under the historical environment, and the next control command whose execution status is determined to be executed or canceled, are obtained. The historical environmental data, the historical control behaviors, the historical control commands, and the next control commands whose execution status is determined to be executed or canceled are used as training data to train the network model until the model converges, thereby obtaining the user habit prediction model.

[0117] Further, the historical environment data, the historical control behaviors, the historical control instructions, and the next control instructions whose execution states are determined to be executed or canceled are used as training data to train the network of the model to be trained, including:

[0118] The historical environment data, historical control behaviors, historical control instructions, and the next control instruction whose execution state is determined to be executed are identified as positive samples. The historical environment data, historical control behaviors, historical control instructions, and the next control instruction whose execution state is cancelled are identified as negative samples. The network training of the model to be trained is performed based on the positive samples and the negative samples.

[0119] Based on the above embodiments, module 330 is specifically used for:

[0120] When the feedback instruction is determined to be positive feedback, the execution state is determined to be "determined to execute"; when the feedback instruction is determined to be negative feedback, the execution state is determined to be "cancelled to execute".

[0121] Based on the above embodiments, the execution module 340 is further configured to:

[0122] When the execution status is determined to be canceled, the vehicle is controlled to maintain its current state of operation.

[0123] The vehicle operation control device provided in the embodiments of the present invention can execute the vehicle operation control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the vehicle operation control method.

[0124] It is worth noting that in the embodiments of the above-mentioned vehicle operation control device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0125] Figure 4 This is a structural schematic diagram of a vehicle provided in an embodiment of the present invention. Figure 4 A block diagram of an exemplary vehicle 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The vehicle 4 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0126] like Figure 4 As shown, vehicle 4 is represented in the form of a general-purpose computing electronic device. The components of vehicle 4 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0127] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0128] Vehicle 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by vehicle 4, including volatile and non-volatile media, removable and non-removable media.

[0129] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Vehicle 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0130] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0131] Vehicle 4 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with vehicle 4, and / or with any device that enables vehicle 4 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, vehicle 4 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 4 As shown, network adapter 20 communicates with other modules of vehicle 4 via bus 18. It should be understood that, although... Figure 4 As not shown in the diagram, other hardware and / or software modules may be used in conjunction with vehicle 4, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0132] Processing unit 16 executes various functional applications and page displays by running programs stored in system memory 28, such as implementing the vehicle operation control method provided in this embodiment of the invention, which includes:

[0133] Acquire current environmental data, as well as the current user's control behavior and commands regarding the vehicle;

[0134] The current environmental data, the control behavior, and the control command are input into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control command and control parameters for the vehicle based on the current environmental data, the control behavior, and the control command.

[0135] The execution status of the predictive control command is determined based on the current user's feedback instructions to the predictive control command;

[0136] When the execution state is determined to be "determined execution", the vehicle's operation is controlled by executing the predictive control command based on the control parameters.

[0137] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the vehicle operation control method provided in any embodiment of the present invention.

[0138] This invention provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements, for example, the vehicle operation control method provided in this invention, the method comprising:

[0139] Acquire current environmental data, as well as the current user's control behavior and commands regarding the vehicle;

[0140] The current environmental data, the control behavior, and the control command are input into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control command and control parameters for the vehicle based on the current environmental data, the control behavior, and the control command.

[0141] The execution status of the predictive control command is determined based on the current user's feedback instructions to the predictive control command;

[0142] When the execution state is determined to be "determined execution", the vehicle's operation is controlled by executing the predictive control command based on the control parameters.

[0143] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0144] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0145] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0146] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0148] Furthermore, the acquisition, storage, use, and processing of data in the technical solution of this invention all comply with relevant laws and regulations.

[0149] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for controlling the operation of a vehicle, characterized in that, include: Acquire current environmental data, as well as the current user's control behavior and commands regarding the vehicle; The current environmental data, the control behavior, and the control command are input into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control command and control parameters for the vehicle based on the current environmental data, the control behavior, and the control command. The execution status of the predictive control command is determined based on the current user's feedback instructions to the predictive control command; When the execution state is determined to be "determined execution", the vehicle's operation is controlled by executing the predictive control command based on the control parameters; The process of constructing the training model corresponding to the user habit prediction model includes: The user's control behavior on the vehicle is modeled as a continuous behavior space, and a Markov process is constructed based on the continuous behavior space. A spatiotemporal feature map is constructed based on the execution time and execution location of each control behavior in the continuous behavior space; The Markov process and the spatiotemporal feature map are combined to construct the model to be trained; The training process of the user habit prediction model includes: The acquisition of historical environmental data, as well as the historical control behaviors and commands of each user for the vehicle under the historical environment, and the execution status of the corresponding historical control commands, determines the next control command to be executed or canceled. The historical environment data, historical control behaviors, historical control instructions, and the next control instructions whose execution states are determined to be executed or canceled are used as training data to train the network of the model to be trained until the model converges, thereby obtaining the user habit prediction model.

2. The vehicle operation control method according to claim 1, characterized in that, After obtaining the current user's control behavior and control commands for the vehicle, it also includes: By comparing and learning, the semantic information of the control behavior and the control instructions is aligned.

3. The vehicle operation control method according to claim 1, characterized in that, The network training of the model to be trained includes using the historical environment data, the historical control behavior, the historical control instructions, and the next control instructions whose execution states are determined to be executed or canceled as training data, including: The historical environmental data, the historical control behavior, the historical control instructions, and the next control instruction whose execution status is determined to be executed are identified as positive samples; the historical environmental data, the historical control behavior, the historical control instructions, and the next control instruction whose execution status is cancelled are identified as negative samples. The network is trained based on the positive and negative samples.

4. The vehicle operation control method according to claim 1, characterized in that, Determining the execution status of the predictive control command based on the current user's feedback instructions to the predictive control command includes: When the feedback instruction is determined to be positive feedback, the execution state is determined to be execution. When the feedback instruction is determined to be negative feedback, the execution status is determined to be canceled.

5. The vehicle operation control method according to claim 1, characterized in that, Also includes: When the execution status is determined to be canceled, the vehicle is controlled to maintain its current state of operation.

6. A vehicle operation control device, characterized in that, include: The acquisition module is used to acquire current environmental data as well as the current user's control behavior and control commands for the vehicle. The prediction module is used to input the current environmental data, the control behavior, and the control command into a pre-trained user habit prediction model, so that the user habit prediction model can determine the current user's predicted control command and control parameters for the vehicle based on the current environmental data, the control behavior, and the control command. The determination module is used to determine the execution status of the predictive control instruction based on the current user's feedback instructions to the predictive control instruction; An execution module is used to control the operation of the vehicle by executing the predictive control command based on the control parameters when the execution state is determined to be determined execution. The process of constructing the training model corresponding to the user habit prediction model includes: The user's control behavior on the vehicle is modeled as a continuous behavior space, and a Markov process is constructed based on the continuous behavior space. A spatiotemporal feature map is constructed based on the execution time and execution location of each control behavior in the continuous behavior space; The Markov process and the spatiotemporal feature map are combined to construct the model to be trained; The training process of the user habit prediction model includes: The acquisition of historical environmental data, as well as the historical control behaviors and commands of each user for the vehicle under the historical environment, and the execution status of the corresponding historical control commands, determines the next control command to be executed or canceled. The historical environment data, historical control behaviors, historical control instructions, and the next control instructions whose execution states are determined to be executed or canceled are used as training data to train the network of the model to be trained until the model converges, thereby obtaining the user habit prediction model.

7. A vehicle, characterized in that, The vehicles include: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the vehicle operation control method as described in any one of claims 1-5.

8. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the vehicle operation control method as described in any one of claims 1-5.

Images