Vehicle control method and device, vehicle, storage medium, program product and chip

Abstract

The present disclosure provides a vehicle control method, device, vehicle, storage medium, program product and chip, and relates to the technical field of intelligent driving. The vehicle control method comprises: acquiring multi-modal perception data of a vehicle, encoding the multi-modal perception data to obtain encoded feature data, and, at any non-first current denoising step in a denoising diffusion process, performing feature sampling according to the encoded feature data and probability information of a previous denoising step to obtain sampling feature data of the current denoising step, wherein the probability information is used to represent probability differences in different reward score trajectory information based on the sampling feature data of the previous denoising step, generating target trajectory information according to the sampling feature data corresponding to multiple denoising steps in the denoising diffusion process, and controlling the vehicle to travel according to the target trajectory information. The technical problem of low trajectory information generation precision in the prior art is solved.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent driving technology, and in particular to a vehicle control method, device, vehicle, storage medium, program product, and chip. Background Technology

[0002] Intelligent driving refers to the ability of a vehicle to drive autonomously without human intervention, achieved through artificial intelligence, sensors, and other technologies. Intelligent driving relies on the collaborative efforts of artificial intelligence, computer vision, radar, monitoring devices, and global positioning systems to enable terminals to automatically and safely operate motor vehicles without any active human intervention. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a vehicle control method, device, vehicle, non-transitory computer-readable storage medium, chip, and computer program product that can adaptively optimize the sampling direction of trajectory information, thereby generating target trajectory information with better reward scores and greatly improving the generation accuracy of target trajectory information.

[0004] According to a first aspect of the present disclosure, a vehicle control method is provided, comprising: acquiring multimodal perception data of a vehicle and encoding the multimodal perception data to obtain encoded feature data; in any non-first current denoising step during the denoising diffusion process, performing feature sampling based on the encoded feature data and probability information of the previous denoising step to obtain sampled feature data for the current denoising step; wherein the probability information is used to represent the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step; generating target trajectory information based on the sampled feature data corresponding to multiple denoising steps during the denoising diffusion process; and controlling the vehicle to drive based on the target trajectory information.

[0005] In the above embodiments, multimodal perception data of the vehicle is acquired and encoded to obtain encoded feature data. During the denoising diffusion process, at any non-first denoising step, feature sampling is performed based on the encoded feature data and the probability information of the previous denoising step to obtain sampled feature data for the current denoising step. The probability information represents the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step. Furthermore, target trajectory information is generated based on the sampled feature data corresponding to multiple denoising steps during the denoising diffusion process, and the vehicle's movement is controlled according to the target trajectory information. Therefore, since "the probability information represents the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step," when the probability information of the previous denoising step is used to guide the sampling direction of the feature sampling in the current denoising step, the sampling direction of the trajectory information can be adaptively optimized, resulting in the generation of target trajectory information with better reward scores, greatly improving the accuracy of target trajectory information generation.

[0006] Optionally, in some embodiments of this disclosure, feature sampling is performed based on the encoded feature data and the probability information of the previous denoising step, including:

[0007] Based on the probability information of the previous denoising step, determine the feature sampling parameters for the current denoising step;

[0008] Based on the feature sampling parameters of the current denoising step, feature sampling is performed on the encoded feature data.

[0009] In the above embodiments, during the process of feature sampling based on encoded feature data and the probability information of the previous denoising step, the feature sampling parameters for the current denoising step can be determined based on the probability information of the previous denoising step, and feature sampling can be performed on the encoded feature data based on the feature sampling parameters of the current denoising step. This improves the precision of feature sampling, enables dynamic adjustment of feature sampling for each denoising step, and enhances the accuracy of feature sampling.

[0010] Optionally, in some embodiments of this disclosure, the method further includes:

[0011] Obtain positive and negative probability information based on the previous denoising step. The positive probability information is the conditional probability information for generating the first trajectory information based on the sampled feature data of the previous denoising step, and the negative probability information is the conditional probability information for generating the second trajectory information based on the sampled feature data of the previous denoising step. The reward score of the first trajectory information is higher than the reward score of the second trajectory information.

[0012] The positive and negative probability information is determined as the probability information of the previous denoising step.

[0013] In the above embodiments, it is possible to accurately optimize the feature sampling of each denoising step step step by step, and guide the sampling direction through positive probability information and negative probability information, thereby improving the sampling accuracy and sampling effect.

[0014] Optionally, in some embodiments of this disclosure, the feature sampling parameters for the current denoising step are determined based on the probability information of the previous denoising step, including:

[0015] Determine the feature sampling parameters of the previous denoising step;

[0016] Based on the positive and negative probability information, the feature sampling parameters of the previous denoising step are adjusted. The feature sampling parameters are used to make the target trajectory information tend to the first trajectory information.

[0017] The adjusted feature sampling parameters are determined as the feature sampling parameters for the current denoising step.

[0018] In the above embodiments, feature sampling in each denoising step can be optimized step by step to guide the sampling direction and ensure that the generated target trajectory information tends to the first trajectory information with a higher reward score.

[0019] Optionally, in some embodiments of this disclosure, the feature sampling parameters of the previous denoising step are adjusted based on positive and negative probability information, including:

[0020] Determine the weight information, which is used to control the intensity of the adjustment of the feature sampling parameters;

[0021] Based on the weight information, positive probability information, and negative probability information, determine the parameter adjustment information;

[0022] The feature sampling parameters of the previous denoising step are adjusted based on the parameter adjustment information.

[0023] In the above embodiments, during the adjustment of the feature sampling parameters of the previous denoising step based on positive and negative probability information, weight information can be determined. This weight information controls the intensity of the feature sampling parameter adjustment. Based on the weight information, positive probability information, and negative probability information, parameter adjustment information is determined, and the feature sampling parameters of the previous denoising step are adjusted according to this information. This enables adaptive adjustment of the feature sampling parameters of the previous denoising step, significantly improving adjustment accuracy and practicality.

[0024] Optionally, in some embodiments of this disclosure, any denoising step of the denoising diffusion process is executed by a prediction model, wherein the prediction model includes: a decoder, a first sub-model connected to the decoder, and a second sub-model.

[0025] The process of obtaining the positive and negative probability information based on the previous denoising step includes:

[0026] In the previous denoising step, the decoder performs feature sampling on the encoded feature data to obtain the sampled feature data of the previous denoising step;

[0027] The first sub-model processes the sampled feature data from the previous denoising step to obtain positive probability information;

[0028] The negative probability information is obtained by processing the sampled feature data from the previous denoising step through the second sub-model;

[0029] The first sub-model has modeled and learned the mapping relationship between the sampled feature data and positive probability information of the previous denoising step, while the second sub-model has modeled and learned the mapping relationship between the sampled feature data and negative probability information of the previous denoising step.

[0030] In the above embodiments, since any denoising step in the denoising diffusion process is completed through a pre-trained prediction model, this prediction model may include: an encoder, a decoder connected to the encoder, and a first sub-model and a second sub-model connected to the decoder. The first sub-model has modeled and learned the mapping relationship between the sampled feature data and positive probability information of the previous denoising step, and the second sub-model has modeled and learned the mapping relationship between the sampled feature data and negative probability information of the previous denoising step. This can greatly improve the prediction accuracy and efficiency of target trajectory information, and is effectively applicable to real-time vehicle driving control, greatly ensuring driving safety.

[0031] Optionally, in some embodiments of this disclosure, the prediction model further includes: an encoder connected to the decoder; wherein, the multimodal sensing data is encoded to obtain encoded feature data, including:

[0032] Encoded feature data is obtained by encoding multimodal sensing data through an encoder.

[0033] In the above embodiments, the efficiency and accuracy of encoding processing can be greatly improved, ensuring that more accurate encoded feature data is extracted, thereby further improving the prediction accuracy and efficiency of target trajectory information.

[0034] Optionally, in some embodiments of this disclosure, acquiring multimodal perception data of the vehicle includes:

[0035] Collect environmental images of the vehicle's driving environment and spatial point cloud data of the driving environment;

[0036] Determine the vehicle's status and navigation information;

[0037] Environmental images, spatial point cloud data, vehicle status information, and navigation information are identified as multimodal perception data.

[0038] In the above embodiments, environmental images and spatial point cloud data of the vehicle's driving environment are collected, and the vehicle's state information and navigation information are determined. The environmental images, spatial point cloud data, vehicle state information, and navigation information are then defined as multimodal perception data. This allows for the collection of more refined multimodal perception data, which can help predict more accurate trajectory information.

[0039] According to a second aspect of the present disclosure, a vehicle control device is provided, comprising: an acquisition unit, configured to acquire multimodal perception data of a vehicle and encode the multimodal perception data to obtain coded feature data; a processing unit, configured to perform feature sampling based on the coded feature data and probability information of the previous denoising step at any non-first current denoising step in the denoising diffusion process to obtain sampled feature data for the current denoising step; wherein the probability information is used to represent the probability difference when generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step; a generation unit, configured to generate target trajectory information based on the sampled feature data corresponding to multiple denoising steps in the denoising diffusion process; and a control unit, configured to control the vehicle to drive based on the target trajectory information.

[0040] Optionally, in some embodiments of this disclosure, the processing unit is further configured to:

[0041] Based on the probability information of the previous denoising step, determine the feature sampling parameters for the current denoising step;

[0042] Based on the feature sampling parameters of the current denoising step, feature sampling is performed on the encoded feature data.

[0043] Optionally, in some embodiments of this disclosure, the processing unit is further configured to:

[0044] Obtain positive and negative probability information based on the previous denoising step. The positive probability information is the conditional probability information for generating the first trajectory information based on the sampled feature data of the previous denoising step, and the negative probability information is the conditional probability information for generating the second trajectory information based on the sampled feature data of the previous denoising step. The reward score of the first trajectory information is higher than the reward score of the second trajectory information.

[0045] The positive and negative probability information is determined as the probability information of the previous denoising step.

[0046] Optionally, in some embodiments of this disclosure, the processing unit is further configured to:

[0047] Determine the feature sampling parameters of the previous denoising step;

[0048] Based on the positive and negative probability information, the feature sampling parameters of the previous denoising step are adjusted. The feature sampling parameters are used to make the target trajectory information tend to the first trajectory information.

[0049] The adjusted feature sampling parameters are determined as the feature sampling parameters for the current denoising step.

[0050] According to a third aspect of the present disclosure, a vehicle is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: implement the steps of the vehicle control method provided in the first aspect of the present disclosure.

[0051] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to execute a vehicle control method. The method includes: acquiring multimodal perception data of a vehicle and encoding the multimodal perception data to obtain encoded feature data; in any non-first current denoising step during the denoising diffusion process, performing feature sampling based on the encoded feature data and probability information of the previous denoising step to obtain sampled feature data for the current denoising step; wherein the probability information is used to represent the probability difference when generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step; generating target trajectory information based on the sampled feature data corresponding to multiple denoising steps during the denoising diffusion process; and controlling vehicle driving based on the target trajectory information.

[0052] According to a fifth aspect of the present disclosure, a chip is provided, the chip including a processing circuit and an interface circuit; wherein the interface circuit is used to read instructions and send instructions to the processing circuit so that the processing circuit executes the vehicle control method as proposed in the first aspect of the present disclosure.

[0053] According to a sixth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the vehicle control method as proposed in the first aspect of the present disclosure.

[0054] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0055] By acquiring and encoding multimodal perception data of the vehicle, coded feature data is obtained. In any non-first denoising step during the denoising diffusion process, feature sampling is performed based on the coded feature data and the probability information from the previous denoising step to obtain the sampled feature data for the current denoising step. The probability information represents the probability difference in generating trajectory information with different reward scores based on the sampled feature data from the previous denoising step. Furthermore, based on the sampled feature data from multiple denoising steps during the denoising diffusion process, target trajectory information is generated, and vehicle movement is controlled according to the target trajectory information. Therefore, since "probability information represents the probability difference in generating trajectory information with different reward scores based on the sampled feature data from the previous denoising step," when the probability information from the previous denoising step is used to guide the sampling direction of the feature sampling in the current denoising step, the sampling direction of the trajectory information can be adaptively optimized, resulting in target trajectory information with better reward scores. This significantly improves the accuracy of target trajectory information generation, better meets actual user needs, and enhances the user experience.

[0056] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0057] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0058] Figure 1 This is a flowchart illustrating a vehicle control method according to some embodiments of the present disclosure;

[0059] Figure 2 This is a flowchart illustrating another vehicle control method according to some embodiments of the present disclosure;

[0060] Figure 3 This is a flowchart illustrating yet another vehicle control method according to some embodiments of the present disclosure;

[0061] Figure 4 This is a structural diagram of a vehicle control device according to some embodiments of the present disclosure;

[0062] Figure 5 This is a functional block diagram of a vehicle shown in an exemplary embodiment;

[0063] Figure 6 This is a schematic diagram of the structure of a chip according to an embodiment of this disclosure;

[0064] Figure 7 This is a schematic diagram of another chip structure proposed in an embodiment of this disclosure. Detailed Implementation

[0065] Some embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0066] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0067] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0068] Figure 1 This is a flowchart illustrating a vehicle control method according to some embodiments of the present disclosure, such as... Figure 1 As shown, the vehicle control method can be used in electronic devices, such as mobile terminals and in-vehicle devices. This vehicle control method can be applied in intelligent driving scenarios or assisted driving scenarios, without limitation. The vehicle control method includes the following steps:

[0069] Step S101: Acquire multimodal perception data of the vehicle and encode the multimodal perception data to obtain encoded feature data.

[0070] Optionally, in some embodiments, the multimodal perception data can be perception data of various modalities collected during vehicle driving. For example, it can be image data of the driving environment collected by an onboard camera, vehicle status data (such as vehicle speed, direction, control parameters, etc.) collected by sensors, and data on obstacles and sound in the driving environment.

[0071] Optionally, in some embodiments, acquiring multimodal perception data of the vehicle includes: acquiring environmental images of the vehicle's driving environment and spatial point cloud data of the driving environment; determining the vehicle's state information and navigation information; and defining the environmental images, spatial point cloud data, vehicle state information, and navigation information as multimodal perception data. This allows for the acquisition of more refined multimodal perception data, which can help predict more accurate trajectory information.

[0072] Optionally, in some embodiments, during the process of acquiring environmental images of the vehicle's driving environment, the environmental images can be captured in real time by an onboard vision sensor (such as a camera or LiDAR), or captured at a certain period. During the process of acquiring spatial point cloud data of the driving environment, a laser pulse can be emitted by an onboard LiDAR and the reflection time measured to generate three-dimensional point cloud data. This three-dimensional point cloud data can be used as spatial point cloud data, capable of describing the spatial characteristics of the vehicle's driving environment in a three-dimensional manner. During the process of determining the vehicle's state information, this state can refer to the vehicle's driving status, such as speed, direction, acceleration, fuel consumption, etc. During the process of determining the vehicle's navigation information, this navigation information can be, for example, navigation data from an onboard device, such as data from a Global Navigation Satellite System (GNSS), without limitation.

[0073] Of course, data from any possible driving environment and vehicle information can be collected during vehicle operation to predict trajectory information, without any restrictions.

[0074] Optionally, in some embodiments, after collecting multimodal perception data of the vehicle, the multimodal perception data can be encoded to obtain encoded feature data. The encoded feature data is used to provide a unified feature representation of the multimodal perception data. For example, the encoded feature data is an encoded feature vector. Alternatively, feature extraction can be performed on the multimodal perception data, and combined with some encoding techniques, the extracted features can be converted into a unified feature representation to obtain the encoded feature data. There are no limitations on this process.

[0075] Step S102: In any non-first current denoising step during the denoising diffusion process, feature sampling is performed based on the encoded feature data and the probability information of the previous denoising step to obtain the sampled feature data of the current denoising step. The probability information is used to represent the probability difference when generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step.

[0076] The denoising diffusion process is a data processing procedure in artificial intelligence, aiming to gradually recover the original structured data from noisy data. The noisy data can be, for example, the original input data, such as the coded feature data in this embodiment, and the output can be the recovered data, such as the target trajectory information in this embodiment. The target trajectory information can be the predicted future trajectory information of the vehicle, used to assist in intelligent or autonomous driving. The denoising diffusion process can be specifically applied in diffusion models in artificial intelligence. This process can be implemented using a reverse Markov chain, forming a symmetrical operation with the forward noise addition process.

[0077] Optionally, the denoising diffusion process can include multiple denoising steps, each of which can be understood as an iterative step, in which the input data is denoised once.

[0078] Optionally, in this embodiment of the present disclosure, the encoded feature data can be processed based on the denoising diffusion process. Each denoising step in the denoising diffusion process can be executed to gradually denoise the encoded feature data and obtain the optimal target trajectory information.

[0079] Optionally, in some embodiments, feature sampling can be performed at any non-first current denoising step during the denoising diffusion process, based on the encoded feature data and the probability information of the previous denoising step. The sampled features can be referred to as the sampled feature data of the current denoising step. That is, each denoising step outputs one sampled feature data, and the multiple sampled feature data output by multiple denoising steps can be used to predict target trajectory information.

[0080] The step of "performing feature sampling based on encoded feature data and the probability information from the previous denoising step" is explained as follows: Each denoising step can output probability information, which represents the probability difference when generating trajectory information with different reward scores based on the sampled feature data of the corresponding denoising step. For example, trajectory information can include at least two types: first trajectory information and second trajectory information. The reward score of the first trajectory information can be higher than that of the second trajectory information. The first trajectory information is, for example, a trajectory information that tends to be safe and reliable, while the second trajectory information is, for example, a trajectory information that is relatively dangerous and unreliable. During the execution of the current denoising step, the probability information from the previous denoising step can be obtained. This probability information represents the probability difference when generating trajectory information with different reward scores based on the sampled feature data from the previous denoising step. The current denoising step has corresponding parameters for feature sampling. Based on the probability information from the previous denoising step, the sampling direction of the current denoising step can be guided and adjusted, that is, the parameters for feature sampling are adjusted, and then feature sampling for the current denoising step is performed based on the adjusted parameters. Furthermore, the aforementioned "probability difference" can guide the generation of target trajectory information that tends towards the first trajectory information, thereby improving the credibility of the target trajectory information and enhancing driving safety.

[0081] Optionally, in some embodiments, for multiple denoising steps in the denoising diffusion process, each denoising step can be executed sequentially. In the first denoising step, feature sampling can be directly performed on the encoded feature data to obtain the sampled feature data of the current denoising step. At the same time, a probability information can be output in this denoising step. Then, the next denoising step is executed, which can be regarded as the "current denoising step". In the "current denoising step", the parameters required for feature sampling in the current denoising step are determined according to the aforementioned "output probability information". Then, feature sampling is performed on the encoded feature data using the "parameters required for feature sampling" to obtain the sampled feature data of the current denoising step. Furthermore, in the current denoising step, a probability information can also be output based on the sampled feature data of the current denoising step. This output probability information is used to perform feature sampling on the encoded feature data in the next denoising step. That is, due to the successive execution of each denoising step, the "next denoising step" can be regarded as the next "current denoising step", and the denoising diffusion process is executed iteratively.

[0082] Optionally, in the process of implementing feature sampling based on encoded feature data and the probability information of the previous denoising step, the feature sampling parameters for the current denoising step can be determined based on the probability information of the previous denoising step, and feature sampling can be performed on the encoded feature data based on the feature sampling parameters of the current denoising step. This improves the fineness of feature sampling, enables dynamic adjustment of feature sampling for each denoising step, and enhances the accuracy of feature sampling.

[0083] For example, the probability information from the previous denoising step represents the probability difference when generating trajectory information with different reward scores based on the sampled feature data from the previous denoising step. This probability difference can be a quantized value. For instance, based on the sampled feature data from the previous denoising step, trajectory information A and trajectory information B can be generated. The reward scores of trajectory information A and trajectory information B are different; for example, the reward score of trajectory information A may be higher than that of trajectory information B. We can determine "probability information A (e.g., probability gradient A, or any other possible quantized probability value) for generating trajectory information A based on the sampled feature data from the previous denoising step," and "probability information B (e.g., probability gradient B, or any other possible quantized probability value) for generating trajectory information B based on the sampled feature data from the previous denoising step." Then, we determine the probability difference between probability information A and probability information B. For example, we can subtract probability information A from probability information B to obtain the probability difference, which serves as the probability information for the previous denoising step. Subsequently, the feature sampling for the current denoising step can be guided based on the probability difference between probability information A and probability information B. For example, the feature sampling parameter P of the current denoising step can be determined based on the probability difference between probability information A and probability information B (i.e., the feature sampling parameter of each denoising step can be determined based on the probability information of the previous denoising step; the feature sampling parameter P can be, for example, the sampling type, sampling granularity, etc., without restriction). Then, the "feature sampling parameter P of the current denoising step" can be used to sample the encoded feature data. The sampled feature data can be called the "sampled feature data of the current denoising step". The above process can be used to obtain the sampled feature data of the corresponding denoising step in any non-first current denoising step during the denoising diffusion process.

[0084] Optionally, in the above process, since "probability information is used to represent the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step", when the probability information of the previous denoising step is used to assist in guiding the sampling direction of the feature sampling of the current denoising step, the sampling direction of the trajectory information can be adaptively optimized, so as to generate target trajectory information with better reward scores and greatly improve the generation accuracy of target trajectory information.

[0085] Step S103: Generate target trajectory information based on the sampling feature data corresponding to multiple denoising steps in the denoising diffusion process.

[0086] Optionally, each denoising step in the above-described denoising diffusion process yields a corresponding sampled feature data. Then, target trajectory information can be generated based on the sampled feature data corresponding to multiple denoising steps in the denoising diffusion process. For example, trajectory prediction can be performed based on multiple sampled feature data to obtain the target trajectory information.

[0087] Step S104: Control the vehicle's movement based on the target trajectory information.

[0088] Optionally, after generating the target trajectory information, the vehicle's movement can be controlled based on this information. For example, driving control commands can be generated based on the target trajectory information, and the vehicle can be controlled using these commands; there are no restrictions on this.

[0089] In this embodiment, multimodal perception data of the vehicle is acquired and encoded to obtain encoded feature data. During the denoising diffusion process, at any non-first denoising step, feature sampling is performed based on the encoded feature data and the probability information of the previous denoising step to obtain sampled feature data for the current denoising step. The probability information represents the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step. Furthermore, target trajectory information is generated based on the sampled feature data corresponding to multiple denoising steps during the denoising diffusion process, and the vehicle's movement is controlled according to the target trajectory information. Therefore, since "the probability information represents the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step," when the probability information of the previous denoising step is used to guide the sampling direction of the feature sampling in the current denoising step, the sampling direction of the trajectory information can be adaptively optimized, resulting in target trajectory information with better reward scores. This greatly improves the accuracy of target trajectory information generation, better meets actual user needs, and enhances the user experience.

[0090] Figure 2 This is a flowchart illustrating another vehicle control method according to some embodiments of the present disclosure, such as... Figure 2 As shown, the vehicle control method can be used in electronic devices, such as mobile terminals and in-vehicle devices. This vehicle control method can be applied in intelligent driving scenarios or assisted driving scenarios, without limitation. The vehicle control method includes the following steps:

[0091] Step S201: Acquire multimodal perception data of the vehicle and encode the multimodal perception data to obtain encoded feature data.

[0092] For a detailed description of S201, please refer to the above embodiments, which will not be repeated here.

[0093] Step S202: Perform the denoising diffusion process to obtain positive probability information and negative probability information based on the previous denoising step. The positive probability information is the conditional probability information for generating the first trajectory information based on the sampled feature data of the previous denoising step, and the negative probability information is the conditional probability information for generating the second trajectory information based on the sampled feature data of the previous denoising step. The reward score of the first trajectory information is higher than the reward score of the second trajectory information.

[0094] For a detailed explanation of the "denoising diffusion process", please refer to the above embodiments.

[0095] Optionally, trajectory information with different reward scores can include: first trajectory information and second trajectory information, where the reward score of the first trajectory information is higher than that of the second trajectory information. Optionally, a higher reward score indicates that the corresponding trajectory information is safer and more reliable, while a lower reward score indicates that the corresponding trajectory information is less safe and more unreliable. For example, the first trajectory information could be the future trajectory information of the vehicle under safe and comfortable driving behavior, and the second trajectory information could be the future trajectory information of the vehicle under risky driving behavior. Furthermore, the positive probability information is the conditional probability information for generating the first trajectory information based on the sampled feature data of the previous denoising step. The negative probability information is the conditional probability information for generating the second trajectory information based on the sampled feature data of the previous denoising step.

[0096] Step S203: Determine the positive probability information and negative probability information as the probability information of the previous denoising step.

[0097] Optionally, the positive and negative probability information obtained from the previous denoising step can be used as the probability information of the previous denoising step.

[0098] Optionally, the positive and negative probability information obtained in the previous denoising step can be used to guide the sampling direction of the feature sampling in the current denoising step, so as to gradually optimize the feature sampling process and ensure that more accurate target trajectory information is generated in the end.

[0099] Step S204: Determine the feature sampling parameters of the previous denoising step.

[0100] Optionally, in each denoising step of the "denoising diffusion process", feature sampling parameters can be used to sample the encoded feature data to obtain the sampled feature data of the corresponding denoising step.

[0101] Optionally, in this embodiment of the disclosure, in the current denoising step, the feature sampling parameters used in the current denoising step can be determined based on the feature sampling parameters used in the previous denoising step.

[0102] Step S205: Adjust the feature sampling parameters of the previous denoising step according to the positive probability information and the negative probability information, and determine the adjusted feature sampling parameters as the feature sampling parameters of the current denoising step. The feature sampling parameters are used to make the target trajectory information tend to the first trajectory information.

[0103] Optionally, in this embodiment of the present disclosure, after determining the feature sampling parameters used in the previous denoising step, the feature sampling parameters of the previous denoising step can be adjusted based on the positive and negative probability information of the previous denoising step. Then, the adjusted feature sampling parameters are determined as the feature sampling parameters of the current denoising step. Therefore, in each denoising step, this method can be used to optimize the feature sampling parameters of the current denoising step. This allows for the gradual optimization of feature sampling in each denoising step, guiding the sampling direction and ensuring that the generated target trajectory information tends towards the first trajectory information with a higher reward score.

[0104] Optionally, in some embodiments, during the adjustment of the feature sampling parameters of the previous denoising step based on positive and negative probability information, weight information can be determined. This weight information controls the intensity of the feature sampling parameter adjustment. Based on the weight information, positive probability information, and negative probability information, parameter adjustment information is determined, and the feature sampling parameters of the previous denoising step are adjusted according to this information. This enables adaptive adjustment of the feature sampling parameters of the previous denoising step, significantly improving adjustment accuracy and practicality.

[0105] For example, the probability difference between positive and negative probability information can be determined, and the probability difference can be weighted using weight information. Based on the weighted result, the feature sampling parameters of the previous denoising step can be adjusted to obtain the feature sampling parameters of the current denoising step. In addition, the "feature sampling parameters of the current denoising step" and the "positive and negative probability information output by the current denoising step" can also be used to determine the "feature sampling parameters of the next denoising step", which will not be elaborated here.

[0106] Optionally, in the process of "determining parameter adjustment information based on weight information, positive probability information, and negative probability information," it can be done by determining a first weight parameter corresponding to the positive probability information and a second weight parameter corresponding to the negative probability information based on the weight information, weighting the positive probability information according to the first weight parameter, weighting the negative probability information according to the second weight parameter, and determining the difference between the weighted positive probability information and the weighted negative probability information, and using this difference as the parameter adjustment information. This allows for the determination of more accurate parameter adjustment information, leading to better feature sampling parameters through optimization.

[0107] For example, the first weight information and the positive probability information can be multiplied to obtain a product result, and the second weight information and the negative probability information can be multiplied to obtain another product result. Then, the difference between the "one product result" and the "other product result" is calculated, and the difference result is used to determine the difference information. This difference information is used as parameter adjustment information to adjust the feature sampling parameters of the previous denoising step. The adjusted feature sampling parameters are used for feature sampling in the current denoising step.

[0108] Step S206: Based on the feature sampling parameters of the current denoising step, perform feature sampling on the encoded feature data to obtain the sampled feature data of the current denoising step.

[0109] Optionally, after obtaining the feature sampling parameters for the current denoising step through the above optimization, feature sampling can be performed on the encoded feature data based on the "feature sampling parameters for the current denoising step" to obtain the sampled feature data for the current denoising step. For example, if feature sampling is performed on the encoded feature data through the decoder, the encoder can be configured using the "feature sampling parameters for the current denoising step," and the configured encoder can be used to perform feature sampling on the encoded feature data to obtain the sampled feature data for the current denoising step; there are no restrictions on this.

[0110] Step S207: Generate target trajectory information based on the sampling feature data corresponding to multiple denoising steps in the denoising diffusion process.

[0111] Step S208: Control the vehicle's movement based on the target trajectory information.

[0112] For a detailed description of S207-S208, please refer to the above embodiments, which will not be repeated here.

[0113] In this embodiment, since "probability information is used to represent the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step," when the probability information of the previous denoising step is used to guide the sampling direction of the feature sampling in the current denoising step, the sampling direction of the trajectory information can be adaptively optimized, resulting in the generation of target trajectory information with a better reward score, thus greatly improving the generation accuracy of the target trajectory information. By acquiring the positive probability information and negative probability information obtained based on the previous denoising step, where the positive probability information is the conditional probability information for generating the first trajectory information based on the sampled feature data of the previous denoising step, and the negative probability information is the conditional probability information for generating the second trajectory information based on the sampled feature data of the previous denoising step, the reward score of the first trajectory information is higher than that of the second trajectory information, and the positive probability information and negative probability information are determined as the probability information of the previous denoising step. The system determines the feature sampling parameters of the previous denoising step and adjusts them based on positive and negative probability information. These parameters are used to guide the target trajectory information towards the first trajectory information. The adjusted parameters are then used as the feature sampling parameters for the current denoising step. These parameters are used to sample the encoded feature data, resulting in the sampled feature data for the current denoising step. This allows for progressive optimization of feature sampling in each denoising step, guiding the sampling direction and ensuring the generated target trajectory information tends towards the first trajectory information with a higher reward score. This also better aligns with actual user needs and improves the user experience. Furthermore, the system enables adaptive adjustment of the feature sampling parameters from the previous denoising step, significantly improving adjustment accuracy and usability. A higher reward score means that, according to user preset settings, the closer the parameters (data) are to the user's desired parameters, the higher the reward score.

[0114] Figure 3 This is a flowchart illustrating yet another vehicle control method according to some embodiments of the present disclosure, such as... Figure 3 As shown, the vehicle control method can be used in electronic devices, such as mobile terminals and in-vehicle devices. This vehicle control method can be applied in intelligent driving scenarios or assisted driving scenarios, without limitation. The vehicle control method includes the following steps:

[0115] Step S301: Acquire multimodal perception data of the vehicle.

[0116] For a detailed description of S301, please refer to the above embodiments, which will not be repeated here.

[0117] Step S302: Perform any denoising step of the denoising diffusion process through the prediction model, wherein the prediction model includes: a decoder, an encoder connected to the decoder, and a first sub-model and a second sub-model connected to the decoder.

[0118] Optionally, in this embodiment, any denoising step in the denoising diffusion process can be implemented using an Artificial Intelligence (AI) model. This AI model is used to predict trajectory information; it can be referred to as a prediction model, and can be a pre-trained model.

[0119] Optionally, the prediction model may include: an encoder, a decoder connected to the encoder, and a first sub-model and a second sub-model connected to the decoder. The first sub-model has modeled and learned the mapping relationship between the sampled feature data and positive probability information from the previous denoising step, and the second sub-model has modeled and learned the mapping relationship between the sampled feature data and negative probability information from the previous denoising step. The encoder can be used to extract and encode features from multimodal perception data to obtain encoded feature data. This encoded feature data can be input into the decoder, which can be used to sample features from the encoded feature data. The backbone network of the decoder can also connect to the first and second sub-models. The first sub-model can be fine-tuned using positive driving behavior, which enhances the generation of future vehicle trajectory information under safe and comfortable driving behavior. The second sub-model can be fine-tuned using negative driving behavior, which enhances the generation of future vehicle trajectory information under risky driving behavior.

[0120] Optionally, during the training of the prediction model, the encoder can be fixed, that is, the encoder is not trained, but the decoder and the first and second sub-models connected to the decoder are trained. The prediction model can be trained on massive human driving data through imitation learning, and its output is the trajectory information of the vehicle in the future. There are no restrictions on this.

[0121] Optionally, the first sub-model described above can be implemented using the low-rank adaptation (LoRA) fine-tuning algorithm. Alternatively, the second sub-model described above can be implemented using the LoRA fine-tuning algorithm; there are no restrictions on which method can be used.

[0122] Step S303: Encode the multimodal sensing data using an encoder to obtain encoded feature data.

[0123] Optionally, during any denoising step of the denoising diffusion process performed by the prediction model, the multimodal sensing data can be encoded by an encoder to obtain encoded feature data. This encoded feature data can be input into the decoder for each denoising step in the denoising diffusion process.

[0124] Step S304: In the previous denoising step, the decoder performs feature sampling on the encoded feature data to obtain the sampled feature data of the previous denoising step.

[0125] Optionally, during the execution of the previous denoising step, the decoder in the prediction model can be used to sample the encoded feature data to obtain the sampled feature data of the previous denoising step. That is, in each denoising step, the decoder is called to sample the encoded feature data output by the encoder to obtain the sampled feature data for the corresponding denoising step. Furthermore, this "sampled feature data from the previous denoising step" can be used to guide the sampling direction of the feature sampling process in the current denoising step.

[0126] Step S305: Process the sampled feature data from the previous denoising step using the first sub-model to obtain positive probability information.

[0127] Step S306: Process the sampled feature data of the previous denoising step through the second sub-model to obtain negative probability information, and determine the positive probability information and negative probability information as the probability information of the previous denoising step.

[0128] Optionally, after performing the previous denoising step and obtaining the sampled feature data of the previous denoising step, the sampled feature data can be further input into the first sub-model and the second sub-model respectively. The first sub-model processes the sampled feature data of the previous denoising step to obtain positive probability information, which is the conditional probability information for generating the first trajectory information based on the sampled feature data of the previous denoising step. The second sub-model processes the sampled feature data of the previous denoising step to obtain negative probability information, which is the conditional probability information for generating the second trajectory information based on the sampled feature data of the previous denoising step. The reward score of the first trajectory information is higher than the reward score of the second trajectory information. Then, the positive probability information and the negative probability information are determined as the probability information of the previous denoising step.

[0129] The construction and training of the above model are explained below:

[0130] For example, multimodal perception data can be collected, which can be represented as s (state). This data describes the state of the driving environment in intelligent driving scenarios (e.g., images captured by onboard cameras or spatial point cloud data). It can also describe the state of the vehicle (e.g., speed, acceleration, driving direction, etc.), navigation information, etc. This data can be used as input to the model, and the model's output can be represented as a (action). This action represents the trajectory of N points corresponding to a fixed time T in the future. For example, 60 coordinate points (6s (i.e., the corresponding time T is 6s), 10Hz (hertz, indicating a sampling frequency of 10Hz)). In other words, the model's output is the vehicle's future trajectory information (an optional example of the aforementioned target trajectory information). During model training, the policy function can be represented as π(a|s), which represents the probability of taking action a given input state s. The reward function used for model training can be represented as R(s,a), which is used to estimate the reward score. The score dimensions can include collision, out of bounds, comfort, driving habits, etc. In this embodiment, state s can be used to represent the above-mentioned "multimodal perception data", and action a can be used to represent the trajectory information predicted above.

[0131] For example, multimodal sensing data can be collected and encoded to obtain encoded feature data. Then, the encoded feature data is input into the model (an optional example of the prediction model mentioned above). The decoder in the model performs feature sampling based on the encoded feature data in the current denoising step to obtain sampled feature data. This sampled feature data can be used to predict trajectory information. The model can further predict trajectory information based on the sampled feature data. Then, the predicted trajectory information is evaluated based on the various scoring dimensions mentioned above to obtain a reward score. Different trajectory information can be predicted, and each trajectory information corresponds to a reward score. The model can also evaluate the probability difference of predicting trajectory information with different reward scores to guide the feature sampling process of the next denoising step until the final target trajectory information is output. In this process, since the trajectory information is scored based on various dimensions such as collision, going out of bounds, comfort, and driving habits, it is ensured that the final output trajectory information can perform well under the aforementioned scoring dimensions (e.g., no collision, no going out of bounds, high comfort, or in line with the user's driving habits, etc.). This comprehensively improves the prediction accuracy and effect of trajectory information, while making the output trajectory information more in line with the user's actual needs and improving the user's driving experience.

[0132] For example, assuming we are currently in the k-th training round, the model obtained from the (k-1)-th optimization round (an optional example of the prediction model above) can be represented as π. (k-1)(a|s), the model obtained from the k-th round of optimization (an optional example of the prediction model above) can be represented as π (k) (a|s). This model can be trained based on imitation learning (encoder + diffuse model decoder). The model may include an encoder and a decoder connected to the encoder. This decoder is based on a diffuse model and can be used to perform the denoising diffuse process. The encoder may not be trained. The diffuse model-based decoder can be divided into two branches, one of which can be represented as... (As an optional example of the first sub-model mentioned above), another branch can be represented as (This is an optional example of the second sub-model described above). The encoder input can be s, and the encoder output can be some representations of s (an optional example of the encoded feature data described above). The decoder based on the diffusion model can receive the representations output by the encoder and output a (an optional example of the predicted trajectory information described above).

[0133] For example, the above The loss function can be expressed as:

[0134]

[0135] The above The loss function can be expressed as:

[0136]

[0137] Here, state s can be used to represent the aforementioned "multimodal sensing data", action a can be used to represent the predicted trajectory information, p(·) is the state distribution of s, and π k-1 (·|s) represents the previous iteration strategy. The reward function used for model training can be represented as R(s,a), where σ(x) represents the sigmoid function. t represents the noisy time step of the diffusion model, T represents the total noisy time step of the diffusion model, z represents Gaussian noise, N(0,I) represents the standard Gaussian distribution, I represents the unit covariance, u represents the uniform distribution of the noisy time steps, and E() represents the expectation.

[0138] In addition, during model training, it is possible to... The loss value output by the loss function (an optional example of the positive probability information mentioned above) and The loss values ​​output by the loss function (an optional example of the negative probability information mentioned above) are weighted and the result can guide the entire model training process. The formula for the weighting operation is as follows:

[0139]

[0140] in, express The loss value output by the loss function. express The loss value output by the loss function, (1+ω) can be an optional example of the first weight parameter mentioned above, and ω can be an optional example of the second weight parameter mentioned above.

[0141] Step S307: In the current denoising step, feature sampling is performed based on the encoded feature data and the probability information of the previous denoising step to obtain the sampled feature data of the current denoising step.

[0142] Step S308: Generate target trajectory information based on the sampling feature data corresponding to multiple denoising steps in the denoising diffusion process.

[0143] Step S309: Control the vehicle's movement based on the target trajectory information.

[0144] For a detailed description of S307-S309, please refer to the above embodiments, which will not be repeated here.

[0145] In this embodiment, since "probability information is used to represent the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step," when the sampling direction of the feature sampling in the current denoising step is guided by the probability information of the previous denoising step, the sampling direction of the trajectory information can be adaptively optimized, resulting in the generation of target trajectory information with better reward scores, thus greatly improving the generation accuracy of the target trajectory information. Furthermore, since any denoising step in the denoising diffusion process is completed through a pre-trained prediction model, this prediction model can include: an encoder, a decoder connected to the encoder, and a first sub-model and a second sub-model connected to the decoder. The first sub-model has modeled and learned the mapping relationship between the sampled feature data and positive probability information of the previous denoising step, and the second sub-model has modeled and learned the mapping relationship between the sampled feature data and negative probability information of the previous denoising step. The encoder is used to encode the multimodal perception data to obtain encoded feature data, which can greatly improve the prediction accuracy and efficiency of the target trajectory information, effectively applying it to real-time vehicle driving control and greatly ensuring driving safety.

[0146] Figure 4 This is a structural diagram of a vehicle control device according to some embodiments of the present disclosure.

[0147] like Figure 4 As shown, the vehicle control device 40 includes:

[0148] The acquisition unit 401 is used to acquire multimodal perception data of the vehicle and encode the multimodal perception data to obtain encoded feature data.

[0149] The processing unit 402 is used to perform feature sampling based on the encoded feature data and the probability information of the previous denoising step in any non-first current denoising step during the denoising diffusion process, to obtain the sampled feature data of the current denoising step. The probability information is used to represent the probability difference when generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step.

[0150] The generation unit 403 is used to generate target trajectory information based on the sampling feature data corresponding to multiple denoising steps in the denoising diffusion process.

[0151] Control unit 404 is used to control the vehicle's movement based on target trajectory information.

[0152] Optionally, in some embodiments of this disclosure, the processing unit 402 is configured to:

[0153] Based on the probability information of the previous denoising step, determine the feature sampling parameters for the current denoising step;

[0154] Based on the feature sampling parameters of the current denoising step, feature sampling is performed on the encoded feature data.

[0155] Optionally, in some embodiments of this disclosure, the processing unit 402 is configured to:

[0156] Obtain positive and negative probability information based on the previous denoising step. The positive probability information is the conditional probability information for generating the first trajectory information based on the sampled feature data of the previous denoising step, and the negative probability information is the conditional probability information for generating the second trajectory information based on the sampled feature data of the previous denoising step. The reward score of the first trajectory information is higher than the reward score of the second trajectory information.

[0157] The positive and negative probability information is determined as the probability information of the previous denoising step.

[0158] Optionally, in some embodiments of this disclosure, the processing unit 402 is configured to:

[0159] Determine the feature sampling parameters of the previous denoising step;

[0160] Based on the positive and negative probability information, the feature sampling parameters of the previous denoising step are adjusted. The feature sampling parameters are used to make the target trajectory information tend to the first trajectory information.

[0161] The adjusted feature sampling parameters are determined as the feature sampling parameters for the current denoising step.

[0162] Optionally, in some embodiments of this disclosure, the processing unit 402 is configured to:

[0163] Determine the weight information, which is used to control the intensity of the adjustment of the feature sampling parameters;

[0164] Based on the weight information, positive probability information, and negative probability information, determine the parameter adjustment information;

[0165] The feature sampling parameters of the previous denoising step are adjusted based on the parameter adjustment information.

[0166] Optionally, in some embodiments of this disclosure, any denoising step of the denoising diffusion process is executed by a prediction model, wherein the prediction model includes: a decoder, a first sub-model connected to the decoder, and a second sub-model.

[0167] The processing unit 402 is used for:

[0168] In the previous denoising step, the decoder performs feature sampling on the encoded feature data to obtain the sampled feature data of the previous denoising step;

[0169] The first sub-model processes the sampled feature data from the previous denoising step to obtain positive probability information;

[0170] The negative probability information is obtained by processing the sampled feature data from the previous denoising step through the second sub-model;

[0171] The first sub-model has modeled and learned the mapping relationship between the sampled feature data and positive probability information of the previous denoising step, while the second sub-model has modeled and learned the mapping relationship between the sampled feature data and negative probability information of the previous denoising step.

[0172] Optionally, in some embodiments of this disclosure, the prediction model further includes: an encoder connected to the decoder; wherein, the acquisition unit 401 is used for:

[0173] Encoded feature data is obtained by encoding multimodal sensing data through an encoder.

[0174] Optionally, in some embodiments of this disclosure, the acquisition unit 401 is used for:

[0175] Collect environmental images of the vehicle's driving environment and spatial point cloud data of the driving environment;

[0176] Determine the vehicle's status and navigation information;

[0177] Environmental images, spatial point cloud data, vehicle status information, and navigation information are identified as multimodal perception data.

[0178] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0179] In this embodiment, multimodal perception data of the vehicle is acquired and encoded to obtain encoded feature data. During the denoising diffusion process, at any non-first denoising step, feature sampling is performed based on the encoded feature data and the probability information of the previous denoising step to obtain sampled feature data for the current denoising step. The probability information represents the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step. Furthermore, target trajectory information is generated based on the sampled feature data corresponding to multiple denoising steps during the denoising diffusion process, and the vehicle's movement is controlled according to the target trajectory information. Therefore, since "the probability information represents the probability difference in generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step," when the probability information of the previous denoising step is used to guide the sampling direction of the feature sampling in the current denoising step, the sampling direction of the trajectory information can be adaptively optimized, resulting in the generation of target trajectory information with better reward scores, greatly improving the accuracy of target trajectory information generation.

[0180] Figure 5 This is a functional block diagram illustrating an exemplary embodiment of a vehicle. For example, vehicle 500 can be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. Vehicle 500 can be an intelligent driving vehicle, a semi-intelligent driving vehicle, or a non-intelligent driving vehicle.

[0181] Reference Figure 5 The vehicle 500 may include various subsystems, such as an infotainment system 510, a perception system 520, a decision control system 530, a drive system 540, and a computing platform 550. The vehicle 500 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of the vehicle 500 can be interconnected via wired or wireless means.

[0182] In some embodiments, the infotainment system 510 may include a communication system, an entertainment system, and a navigation system, etc.

[0183] The perception system 520 may include several sensors for sensing information about the environment surrounding the vehicle 500. For example, the perception system 520 may include a global positioning system (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit, lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0184] The decision control system 530 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0185] The drive system 540 may include components that provide powered motion to the vehicle 500. In one embodiment, the drive system 540 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0186] Some or all of the functions of vehicle 500 are controlled by computing platform 550. Computing platform 550 may include at least one processor 551 and memory 552, and processor 551 may execute instructions 553 stored in memory 552.

[0187] The processor 551 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application-Specific Integrated Circuit (ASIC), or a combination thereof.

[0188] The memory 552 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0189] In addition to instruction 553, memory 552 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 552 can be used by computing platform 550.

[0190] In this embodiment of the disclosure, the processor 551 may execute instructions 553 to complete all or part of the steps of the vehicle control method described above.

[0191] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the vehicle control method provided in this disclosure.

[0192] To implement the above embodiments, this disclosure also proposes a chip, including: the chip includes processing circuitry configured to perform the methods provided in the foregoing embodiments.

[0193] Figure 6 This is a schematic diagram of the structure of a chip according to an embodiment of this disclosure. See also... Figure 6 The diagram shown is a schematic representation of the structure of chip 600, but it is not limited to this.

[0194] Chip 600 includes processing circuit 601 and interface circuit 602. Interface circuit 602 is used to read instructions and send instructions to processing circuit 601 so that processing circuit 601 executes the method in the above embodiment.

[0195] Optionally, such as Figure 7 As shown, Figure 7 This is a schematic diagram of another chip structure proposed in an embodiment of this disclosure. Chip 600 may further include: a memory 603 for storing instructions, and an interface circuit 602 for reading the instructions stored in the memory 603.

[0196] Optionally, the interface circuit 602 is connected to the memory 603. The interface circuit 602 can be used to receive signals from the memory 603 or other devices, and can also be used to send signals to the memory 603 or other devices. For example, the interface circuit 602 can read instructions stored in the memory 603 and send those instructions to the processing circuit 601.

[0197] Optionally, the number of memories 603 can be one or more. The number of interface circuits 602 can also be one or more.

[0198] In some embodiments, the interface circuit 602 performs at least one of the communication steps such as sending and / or receiving in the above method, and the processing circuit 601 performs other steps.

[0199] In some embodiments, the terms interface circuit, interface, transceiver pin, transceiver, etc., can be used interchangeably.

[0200] Alternatively, all or part of the memory 603 may be located outside of the chip 600.

[0201] To implement the above embodiments, this disclosure also proposes a computer program product that, when instructions in the computer program product are executed by a processor, performs the method proposed in the foregoing embodiments of this disclosure.

[0202] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0203] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0204] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0205] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0206] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other. As used herein, the term "and / or" includes any one of the relevant listed items and any combination of any two or more; similarly, "at least one of..." includes any one of the relevant listed items and any combination of any two or more. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description herein, "a plurality of" means at least two, such as two, three, etc., unless otherwise expressly specified.

Claims

1. A vehicle control method, characterized in that, include: Acquire multimodal perception data of the vehicle and encode the multimodal perception data to obtain coded feature data; In any non-first current denoising step during the denoising diffusion process, feature sampling is performed based on the encoded feature data and the probability information of the previous denoising step to obtain the sampled feature data of the current denoising step; wherein, the probability information is used to represent the probability difference when generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step. Based on the sampled feature data corresponding to multiple denoising steps in the denoising diffusion process, target trajectory information is generated; The vehicle is controlled to move according to the target trajectory information; The probability information is determined in the following way: Trajectory information is obtained based on the sampled feature data of the previous denoising step, and the trajectory information is evaluated according to a preset scoring dimension to obtain the probability difference of the trajectory information with different reward scores, which is used as the probability information. The step of performing feature sampling based on the encoded feature data and the probability information of the previous denoising step to obtain the sampled feature data of the current denoising step includes: Based on the sampling direction indicated by the probability information, feature sampling is performed on the encoded feature data to obtain the sampled feature data of the current denoising step.

2. The method according to claim 1, characterized in that, The feature sampling based on the encoded feature data and the probability information from the previous denoising step includes: Based on the probability information of the previous denoising step, determine the feature sampling parameters of the current denoising step; Based on the feature sampling parameters of the current denoising step, feature sampling is performed on the encoded feature data.

3. The method according to claim 2, characterized in that, The method further includes: Obtain positive probability information and negative probability information based on the previous denoising step, wherein the positive probability information is the conditional probability information for generating first trajectory information based on the sampled feature data of the previous denoising step, and the negative probability information is the conditional probability information for generating second trajectory information based on the sampled feature data of the previous denoising step, and the reward score of the first trajectory information is higher than the reward score of the second trajectory information. The positive and negative probability information are determined as the probability information of the previous denoising step.

4. The method according to claim 3, characterized in that, The step of determining the feature sampling parameters for the current denoising step based on the probability information of the previous denoising step includes: Determine the feature sampling parameters of the previous denoising step; Based on the positive probability information and the negative probability information, the feature sampling parameters of the previous denoising step are adjusted, wherein the feature sampling parameters are used to make the target trajectory information tend to the first trajectory information; The adjusted feature sampling parameters are determined as the feature sampling parameters for the current denoising step.

5. The method according to claim 4, characterized in that, The step of adjusting the feature sampling parameters of the previous denoising step based on the positive and negative probability information includes: Determine weight information, wherein the weight information is used to control the intensity of the adjustment of the feature sampling parameters; Based on the weight information, the positive probability information, and the negative probability information, the parameter adjustment information is determined; The feature sampling parameters of the previous denoising step are adjusted according to the parameter adjustment information.

6. The method according to claim 3, characterized in that, Each denoising step in the denoising diffusion process is executed through a prediction model, wherein the prediction model includes: a decoder, a first sub-model and a second sub-model connected to the decoder; The step of obtaining the positive and negative probability information based on the previous denoising step includes: In the previous denoising step, the decoder performs feature sampling on the encoded feature data to obtain the sampled feature data of the previous denoising step. The positive probability information is obtained by processing the sampled feature data of the previous denoising step through the first sub-model; The negative probability information is obtained by processing the sampled feature data of the previous denoising step through the second sub-model; The first sub-model has modeled and learned the mapping relationship between the sampled feature data of the previous denoising step and the positive probability information, and the second sub-model has modeled and learned the mapping relationship between the sampled feature data of the previous denoising step and the negative probability information.

7. The method according to claim 6, characterized in that, The prediction model further includes: an encoder connected to the decoder; wherein, encoding the multimodal sensing data to obtain encoded feature data includes: The encoder encodes the multimodal sensing data to obtain the encoded feature data.

8. The method according to any one of claims 1-7, characterized in that, The acquisition of multimodal perception data of the vehicle includes: Collect environmental images of the vehicle's driving environment and spatial point cloud data of the driving environment; Determine the vehicle's status information and navigation information; The environmental image, the spatial point cloud data, the vehicle's status information, and the navigation information are identified as the multimodal perception data.

9. A vehicle control device, characterized in that, include: An acquisition unit is used to acquire multimodal perception data of a vehicle and encode the multimodal perception data to obtain encoded feature data. The processing unit is configured to perform feature sampling based on the encoded feature data and the probability information of the previous denoising step in any non-first current denoising step during the denoising diffusion process, to obtain the sampled feature data of the current denoising step; wherein, the probability information is used to represent the probability difference when generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step. The generation unit is used to generate target trajectory information based on the sampled feature data corresponding to multiple denoising steps in the denoising diffusion process; A control unit is used to control the vehicle's movement based on the target trajectory information; The processing unit is further configured to: Trajectory information is obtained based on the sampled feature data of the previous denoising step, and the trajectory information is evaluated according to a preset scoring dimension to obtain the probability difference of the trajectory information with different reward scores, which is used as the probability information. Based on the sampling direction indicated by the probability information, feature sampling is performed on the encoded feature data to obtain the sampled feature data of the current denoising step.

10. The apparatus according to claim 9, characterized in that, The processing unit is further configured to: Based on the probability information of the previous denoising step, determine the feature sampling parameters of the current denoising step; Based on the feature sampling parameters of the current denoising step, feature sampling is performed on the encoded feature data.

11. The apparatus according to claim 10, characterized in that, The processing unit is further configured to: Obtain positive probability information and negative probability information based on the previous denoising step, wherein the positive probability information is the conditional probability information for generating first trajectory information based on the sampled feature data of the previous denoising step, and the negative probability information is the conditional probability information for generating second trajectory information based on the sampled feature data of the previous denoising step, and the reward score of the first trajectory information is higher than the reward score of the second trajectory information. The positive and negative probability information are determined as the probability information of the previous denoising step.

12. The apparatus according to claim 11, characterized in that, The processing unit is further configured to: Determine the feature sampling parameters of the previous denoising step; Based on the positive probability information and the negative probability information, the feature sampling parameters of the previous denoising step are adjusted, wherein the feature sampling parameters are used to make the target trajectory information tend to the first trajectory information; The adjusted feature sampling parameters are determined as the feature sampling parameters for the current denoising step.

13. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: The steps of implementing the method according to any one of claims 1 to 8.

14. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a mobile terminal, enable the mobile terminal to perform a method comprising: Acquire multimodal perception data of the vehicle and encode the multimodal perception data to obtain coded feature data; In any non-first current denoising step during the denoising diffusion process, feature sampling is performed based on the encoded feature data and the probability information of the previous denoising step to obtain the sampled feature data of the current denoising step; wherein, the probability information is used to represent the probability difference when generating trajectory information with different reward scores based on the sampled feature data of the previous denoising step. Based on the sampled feature data corresponding to multiple denoising steps in the denoising diffusion process, target trajectory information is generated; The vehicle is controlled to move according to the target trajectory information; The probability information is determined in the following way: Trajectory information is obtained based on the sampled feature data of the previous denoising step, and the trajectory information is evaluated according to a preset scoring dimension to obtain the probability difference of the trajectory information with different reward scores, which is used as the probability information. The step of performing feature sampling based on the encoded feature data and the probability information of the previous denoising step to obtain the sampled feature data of the current denoising step includes: Based on the sampling direction indicated by the probability information, feature sampling is performed on the encoded feature data to obtain the sampled feature data of the current denoising step.

15. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.

16. A chip, said chip comprising processing circuitry and interface circuitry; wherein, The interface circuit is used to read instructions and sends the instructions to the processing circuit so that the processing circuit performs the method as described in any one of claims 1-8.

Images