Training method of decision planning model, vehicle decision planning method and device

Abstract

The application provides a decision planning model training method, a vehicle decision planning method and equipment. The decision planning model training method comprises the following steps: generating input data of a decision planning model to be trained and a weight analysis network based on a two-dimensional label of a scene category in first training data and perception information collected by at least one sensor of a real vehicle; the two-dimensional label of the scene category is used to label each basic scene and a corresponding dynamic interaction information corresponding scene category; the input data is input into the decision planning model to be trained and the weight analysis network respectively, so that the decision planning model outputs a decision result under the scene category, and the weight analysis network learns the two-dimensional label of the scene category and the perception information, and outputs a loss term weight dynamically matched with the decision result; and a trained decision planning model is obtained based on a loss term and the matched loss term weight.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, and in particular to a training method for a decision planning model, a vehicle decision planning method, and a device. Background Technology

[0002] In recent years, autonomous driving has moved from laboratory theoretical verification to engineering mass production and commercial application. Its functional level is also upgrading from low-level assisted driving to L3 (Level 3, conditional autonomous driving) and L4 (Level 4, highly automated driving) level high-level autonomous driving.

[0003] In autonomous driving systems, models rely on scene labels for decision-making and planning. Scene labels set by humans in related technologies all depend to varying degrees on the human experience of engineers.

[0004] Thus, this model is not only inefficient and highly subjective, but also struggles to cover the vast and complex sub-scenarios in the field of autonomous driving, and is difficult to adapt to the decision-making needs of more scenarios. Summary of the Invention

[0005] This application provides an improved method for training a decision planning model, a vehicle decision planning method, and an apparatus.

[0006] This application provides a method for training a decision planning model, including: Based on the two-dimensional labels of scene categories in the first training data, and the perception information collected by at least one sensor of the actual vehicle, input data for the decision planning model to be trained and the weight parsing network are generated; the two-dimensional labels of scene categories are used to label each basic scene and its corresponding dynamic interaction information corresponding to the scene category. The input data is respectively input into the decision planning model to be trained and the weight parsing network, so that the decision planning model outputs the decision result under the scene category, and the weight parsing network learns the two-dimensional label of the scene category and the perception information, and outputs the loss term weights that are dynamically matched with the decision result; Based on the loss term and the weights of the matched loss term, a well-trained decision planning model is obtained.

[0007] Furthermore, the weight parsing network learns the two-dimensional labels of the scene category and the perceptual information, and outputs loss term weights that are dynamically matched with the decision result, including: The weight parsing network deeply integrates the two-dimensional labels of the scene category with the perceived information to generate differentiated weights that adapt to different environmental details of the same scene category, so as to obtain loss term weights that dynamically match the decision result.

[0008] Furthermore, the weight parsing network includes a graph neural network; The step of deeply fusing the two-dimensional labels of the scene category with the perceived information to generate differentiated weights that adapt to different environmental details of the same scene category, and using these weights as loss term weights that are dynamically matched with the decision result, includes: The two-dimensional labels, static elements, and dynamic elements of the scene categories in the perceived information are used to construct an association graph. The graph nodes of the association graph include scene label nodes for representing scene category attributes, as well as static element nodes and dynamic element nodes. The edge weights of the association graph are constructed based on the matching degree between the graph nodes and the scene, as well as the spatial conflict relationship and / or motion conflict relationship between the graph nodes. By dynamically updating the edge weights, we can obtain the perceived changes in different environmental details. Global features that combine scene category attributes and environmental details from the same scene are used to generate differentiated weights.

[0009] Furthermore, the process of fusing global features of scene category attributes and environmental details from the same scene to generate differentiated weights includes: By aggregating the dynamic feature feedback and preference guidance of all nodes through neighborhood features, the system outputs global features that integrate scene category attributes and environmental details of the same scene, generating differentiated weights.

[0010] Furthermore, the number of loss item weights dynamically matched with the decision result is determined in real time based on the scenario category; When there are multiple loss item weights that are dynamically matched with the decision result, the loss item includes multiple loss items accordingly; The weight parsing network learns the two-dimensional labels of the scene category and the perceptual information, and outputs loss term weights that are dynamically matched with the decision result, including: The weight parsing network learns the two-dimensional labels of the scene category and the perception information, and outputs the weights of each loss term that are dynamically matched with the decision result; The process of obtaining the trained decision planning model includes: determining the loss value of each loss term between the decision result and the actual result according to each loss term and the weight of each loss term dynamically matched with the decision result, obtaining the total loss value, and stopping the iteration when the total loss value converges to a stable state, thereby obtaining the trained decision planning model.

[0011] Furthermore, the number of loss items is determined in real time based on the scenario category; When the number of loss term weights dynamically matched with the decision result is 1, the corresponding loss term includes one loss term; The trained decision planning model, based on the loss term and the matched loss term weights, includes: Based on a loss term and a loss term weight that is dynamically matched with the decision result, the loss value of the loss term between the decision result and the actual result is determined respectively. When the loss value converges to a stable state, the iteration stops, and the trained decision programming model is obtained.

[0012] Furthermore, the input data for generating the decision-making and planning model to be trained and the weight parsing network based on the two-dimensional labels of scene categories in the first training data and the perception information collected by at least one sensor of the actual vehicle in the first training data includes: Based on the two-dimensional labels of scene categories in the first training data and the encoded features of the perceived information, a fusion feature containing environmental information and scene context is obtained. The fused features are then input into the decision planning model and the weight parsing network to be trained, respectively.

[0013] This application provides a vehicle decision-making and planning method, including: Acquire current perception information in the current scene collected by the vehicle's sensors; Based on the current perceived information, the current scene category is identified; The current scene category and the current perception information are input into the trained decision planning model, so that the trained decision planning model outputs planning control instructions that are adapted to the current scene category according to the decision preference of the current scene category; the trained decision planning model is a decision planning model trained using the decision planning model training method described above.

[0014] Furthermore, identifying the current scene category based on the current perceived information includes: Acquire current perception information in the current scene collected by the vehicle's sensors; The current perceived information is input into the scene recognition model to output the scene category probability distribution; the scene recognition model is trained based on training data and two-dimensional labels of each basic scene and its corresponding dynamic interaction information corresponding to the scene category. Based on the probability distribution of the scene categories, the current scene category containing the basic scene and its corresponding dynamic interaction information is determined.

[0015] Furthermore, determining the current scene category containing the basic scene and its corresponding dynamic interaction information based on the scene category probability distribution includes: The scene category with the highest probability in the scene category probability distribution is taken as the current scene category.

[0016] Furthermore, the step of taking the scene category with the highest probability in the scene category probability distribution as the current scene category includes: when there are multiple scene categories with the highest probability in the scene category probability distribution, taking any scene category with the highest probability as the current scene category; or, The step of taking the scene category with the highest probability in the scene category probability distribution as the current scene category includes: when there are multiple scene categories with the highest probability in the scene category probability distribution, taking the scene category with the highest security weight among the multiple scene categories with the highest probability as the current scene category.

[0017] Furthermore, determining the current scene category containing the basic scene and its corresponding dynamic interaction information based on the scene category probability distribution includes: taking the scene category corresponding to the scene category probability greater than the confidence threshold as the current scene category; or, Based on the scene category probability distribution, the current scene category containing the basic scene and its corresponding dynamic interaction information is determined, including: If the scene category is determined to be a scene category that is less than the confidence threshold, then the scene category that is less than the confidence threshold is taken as the current scene category.

[0018] Furthermore, the method also includes: the scene recognition model is trained in the following manner: Acquire second training data; the second training data includes: perception information collected by at least one sensor of the actual vehicle, and used to label each basic scene with a two-dimensional label corresponding to the scene category of the dynamic interaction information; The perceived information and the two-dimensional label of the scene category are input into the scene recognition model to be trained, so that the scene recognition model to be trained learns the scene category label of the second training data, and a trained scene recognition model is obtained, which is used as the scene recognition model.

[0019] Furthermore, the scene recognition model includes a feature extraction backbone network and a scene classification head; Environmental features are extracted from the perceived information through the feature extraction backbone network. Based on the environmental features, the scene classification head outputs the probability distribution of the scene category.

[0020] Furthermore, the step of extracting environmental features from the perceived information through the feature extraction backbone network includes: Through the feature extraction backbone network, the static element tensor and dynamic element tensor of the current scene are extracted from the perceived information; The step of outputting a scene category probability distribution based on the environmental features using the scene classification head includes: The static element tensor and the dynamic element tensor are encoded respectively to obtain static features and dynamic features; The static features and the dynamic features are fused to obtain the fused features; The fused features are mapped to a probability distribution of scene categories.

[0021] Furthermore, the encoding of the static element tensor and the dynamic element tensor respectively to obtain static features and dynamic features includes: Based on the attention mechanism, the association weights between the static features and the dynamic features are determined; Based on the association weights, attention weights are applied to the static features and the dynamic features to obtain weighted static features and weighted dynamic features. The weighted static features and the weighted dynamic features are fused to obtain the fused features.

[0022] Furthermore, the acquisition of current perception information in the current scene collected by the vehicle's sensors includes: Acquire current perception information in the current scene from multiple sensors of the actual vehicle; the current perception information includes the current scene and the current dynamic interaction information in the current scene. The step of inputting the current perceived information into the scene recognition model to output a scene category probability distribution includes: The current scene and the current dynamic interaction information from the multiple sensors are synchronously input into the scene recognition model to output the scene category probability distribution.

[0023] This application provides an electronic device, including one or more processors, for implementing the training method of the decision planning model as described above or the vehicle decision planning method as described above.

[0024] This application provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the training method for the decision planning model as described above or the vehicle decision planning method as described above.

[0025] This application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the method described in any of the preceding claims.

[0026] In some embodiments, the training method of the decision planning model of this application achieves fully data-driven adaptive weight generation by constructing a weight parsing network with two-dimensional labels of scene categories and perceptual information as dual inputs, completely eliminating the dependence of related technologies on engineers' human experience. Furthermore, the weight parsing network of this invention can autonomously learn the correlation logic between the two-dimensional labels of scene categories, perceptual information, and loss term weights through massive training data; simultaneously, it dynamically generates weights based on real-time input scene categories and perceptual information, enabling it to adapt to the decision-making needs of various sub-scenarios, effectively overcoming the bottleneck of limited generalization ability in traditional manual solutions, and ensuring full-scene decision adaptability.

[0027] In other embodiments, the vehicle decision-making and planning method, through training data and two-dimensional labels of each basic scene and its corresponding dynamic interaction information corresponding to the scene category, trains a scene recognition model. After recognizing the basic scene and its dynamic interaction information, the trained decision-making and planning model can output planning and control commands adapted to the current scene category. This quickly distinguishes the interactive actions of the current scene, thereby improving the understanding of complex scenes and obtaining planning and control commands adapted to the current scene category, which is beneficial for more precise control. Attached Figure Description

[0028] Figure 1 The diagram shown is an overall flowchart of the vehicle decision-making and planning method according to an embodiment of this application. Figure 2 The diagram shown is a flowchart illustrating the training method for the decision planning model provided in an embodiment of this application. Figure 3 The diagram shown is a schematic diagram of the joint training of the decision planning model and the weight parsing network provided in the embodiment of this application; Figure 4 The diagram shown is a flowchart illustrating the vehicle decision-making and planning method provided in an embodiment of this application. Figure 5 The diagram shown is a flowchart illustrating step 202 of the vehicle decision-making and planning method provided in this embodiment of the application. Figure 6 As shown Figure 5 The diagram shows the training process of the scene recognition model in the vehicle decision-making and planning method. Figure 7 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0030] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0031] Figure 1 The diagram shown is an overall flowchart of the vehicle decision-making and planning method according to an embodiment of this application.

[0032] like Figure 1 As shown, firstly, perception information for each scene category is acquired. This perception information refers to real-vehicle sensor data with scene labels, which includes perception information collected by at least one sensor of the real vehicle, as well as two-dimensional labels used to annotate each basic scene and its corresponding dynamic interaction information corresponding to the scene category. The above perception information can be used as the second training data or as the first training data. The "second" in "second training data" and the "first" in "first training data" distinguish the training data of the scene recognition model to be trained and the decision planning model to be trained.

[0033] Based on this, using the first training data, the decision-making and planning model to be trained can be trained during the offline training phase, enabling the trained model to output dynamic decisions adapted to specific scene categories. Simultaneously, using the second training data, the scene recognition model to be trained can be trained during the offline training phase, enabling this trained model to output specific scene categories.

[0034] The aforementioned decision-making and planning model is used to make inference decisions based on preset scenario decision preferences and fusion features, and to generate planning and control instructions that meet the needs of the current scenario. For example, the aforementioned decision-making and planning model may include, but is not limited to, a PDP (Prediction Decision and Planning) model.

[0035] continue Figure 1As shown, during the offline training phase, the training of the scene recognition model to be trained and the joint training of the weight parsing network and the PDP model to be trained are completed simultaneously. The results are a trained scene recognition model for outputting standardized scene labels, a weight parsing network for generating adaptive loss term weights by fusing scene and perception information, and a trained PDP model for making decisions using standardized scene labels as guidance.

[0036] Finally, the trained scene recognition model and the trained PDP model are deployed collaboratively to construct an integrated online inference system that integrates the trained scene recognition model and the trained PDP model, enabling dynamic decision-making in real-vehicle scenarios.

[0037] Thus, during the offline training phase, a loss guide is constructed to establish the correlation between basic scene categories, perceptual detail information, and loss term weights. During the online phase, the PDP model, already trained under adaptive loss guidance, outputs planning and control decisions adapted to the current scene category in real time.

[0038] The construction of scene labels for related technologies not only relies on human experience, but also fails to capture the subtle differences in perceptual environments within the same type of scene (such as the density of pedestrians versus the absence of pedestrians in a straight-ahead scene on an urban road). This results in overly coarse-grained weight generation, unable to adapt to subtle changes within the same scene, and still exhibiting a disconnect between decision preferences and actual environmental requirements.

[0039] To address the technical problems of the aforementioned technologies, such as reliance on human experience, inability to adapt to subtle changes within the same scenario, disconnect between decision preferences and actual environmental requirements, difficulty in covering the massive and complex sub-scenarios in the field of autonomous driving, and inability to adapt to decision-making needs in more scenarios, this application provides a training method for a decision planning model. This method uses training data and two-dimensional labels representing each basic scenario and its corresponding dynamic interaction information and scenario category as inputs to the decision planning model to be trained and the weight parsing network. The weight parsing network learns the scenario category, dynamic interaction information, and loss term weights, and outputs loss term weights that dynamically match the decision result, thereby training the decision planning model to be trained.

[0040] In this way, adaptive dynamic optimization of the loss term weights can be achieved, enabling the solution to adapt to more basic scenarios and subdivided scenarios with dynamic interaction information, meeting the needs of more scenarios, thereby improving the generalization ability of the decision-making and planning model and adapting to the decision-making needs of all scenarios. Furthermore, it can quickly distinguish interactive actions, adapt to detailed changes within the same scenario, and ensure that decision preferences match the actual environmental requirements, thereby improving the understanding of complex scenarios and making subsequent decisions more accurate.

[0041] Furthermore, this application provides a training method for a decision planning model. By constructing a weight parsing network with two-dimensional labels of scene categories and perceptual information as dual inputs, it achieves fully data-driven adaptive weight generation, completely eliminating the reliance on engineers' manual experience in related technologies. Moreover, the weight parsing network of this application can autonomously learn the correlation logic between the two-dimensional labels of scene categories, perceptual information, and loss term weights through massive training data. Simultaneously, it dynamically generates weights based on real-time input scene and perceptual information, adapting to the decision-making needs of various sub-scenarios, effectively overcoming the bottleneck of limited generalization ability in traditional manual solutions, and ensuring full-scene decision adaptability.

[0042] Figure 2 The diagram shown is a flowchart illustrating the training method for the decision planning model provided in an embodiment of this application.

[0043] like Figure 2 As shown, the training method for this decision-making and programming model may include, but is not limited to, the following steps 101 to 103: Step 101: Based on the two-dimensional labels of scene categories in the first training data and the perception information collected by at least one sensor of the actual vehicle, generate input data for the decision planning model to be trained and the weight parsing network; the two-dimensional labels of scene categories are used to label each basic scene and its corresponding dynamic interaction information corresponding to the scene category.

[0044] The perceived information is used to represent the actual situation of the current scene, such as the basic scene category and perceived detail information. This perceived information refers to information representing at least two dimensions of the basic scene and its corresponding dynamic interaction information.

[0045] Therefore, the subdivided scenes formed by combining detailed road segment information represented by the basic scene and dynamic interaction information are used as the scene categories in this application's embodiments. Details are as follows: The aforementioned basic scenarios may include, but are not limited to, one or a combination of road environment features such as road type (e.g., highways, urban expressways, ordinary urban roads, rural roads, etc.) and road topology (e.g., straight roads, curves, ramps, intersections, roundabouts, tunnels, bridges, etc.). These basic scenarios are used to represent detailed road segment information, such as at least one of the following: straight sections of urban roads, intersections, ramp sections, and construction sections. Of course, the above are merely illustrative examples; any road environment features that can reflect the current basic scenarios are within the scope of protection of this application and are not specifically limited herein.

[0046] The aforementioned dynamic interaction information is used to characterize the perceived details of the behavioral interactions between various traffic participants in the basic scenario. This dynamic interaction information may include, but is not limited to, at least one of the following: the motion state of traffic participants and the state information of traffic control facilities (such as traffic lights and traffic signs and markings).

[0047] The movement states of the aforementioned traffic participants may include the driving state of vehicles (such as acceleration, deceleration, constant speed, stationary, turning, changing lanes and making a U-turn, etc. at least one), the movement state of pedestrians (such as walking, running, standing and crossing the road, etc. at least one), and the driving state of non-motorized vehicles (such as riding, stopping and turning, etc. at least one).

[0048] Specifically, as shown in the scene category annotation results in Table 1 below, the aforementioned dynamic interaction information includes, but is not limited to, ghost peeks (VRU (Vulnerable Road User)), vehicles cutting in from the adjacent lane, being overtaken, the vehicle in front being a slow-moving large vehicle, detouring around static obstacles, a vehicle going straight and another vehicle turning right, a vehicle turning right and another vehicle going straight, a vehicle turning left and another vehicle turning right, a vehicle turning right and another vehicle turning left, multiple vehicles turning side by side, a vehicle merging from a ramp into the main road, a vehicle cutting into a ramp from the main road, another vehicle cutting into a ramp, a vehicle approaching a construction sign, and multiple vehicles merging into a narrow passage. Of course, the above are merely illustrative examples; any dynamic interaction information that reflects the current basic scenario is within the scope of protection of this application, and no specific limitations are made here.

[0049] Table 1. Annotation results of scene categories

[0050] In this way, dynamic interactive information and basic scenarios together constitute the complex behavioral relationships among traffic participants in the current scenario, making it easy to grasp the dynamic changes within the scenario category.

[0051] The basic scene categories mentioned above are macro-level classifications, while perceived detail information is a micro-level variable. Scene categories, such as straight-ahead driving on urban roads and VRU crossing, are macro-level classification dimensions, corresponding to the labeled type results. However, within the same basic scene category, perceived detail information is a continuous / discrete micro-level variable. For example, the distance between a pedestrian and a vehicle could be 1m, 3m, or 5m; the relative speed of a neighboring vehicle could be -5m / s, 0m / s, or 10m / s.

[0052] Step 102: Input the input data into the decision planning model and the weight parsing network to be trained, so that the decision planning model outputs the decision results under the scene category, and the weight parsing network learns the two-dimensional labels and perception information of the scene category, and outputs the loss term weights that are dynamically matched with the decision results.

[0053] Figure 3The diagram shown is a schematic diagram of the joint training of the decision planning model and the weight parsing network provided in the embodiment of this application.

[0054] like Figure 3 As shown, the weight parsing network is used to adaptively learn and determine the weight allocation of different loss terms during the model training process based on the scene category and dynamic interaction information in the input data.

[0055] Specifically, the weighted parsing network analyzes the road environment characteristics (such as road type, topology, etc.) and dynamic interaction information (such as the movement status of traffic participants, the status of traffic control facilities, etc.) of the basic scenario in the current scenario, and identifies the key factors that have a significant impact on the decision-making and planning results.

[0056] For example, in scenarios involving dynamic interactive information such as vehicles suddenly appearing out of nowhere or cutting in front of others, the weighted parsing network increases the weights of loss terms related to collision risk to enhance the model's learning of such dangerous scenarios.

[0057] For example, in relatively stable basic scenarios such as straight driving on urban roads, the weight of certain loss items may be appropriately reduced, and the focus may be on optimizing objectives related to driving smoothness.

[0058] continue Figure 3 As shown, the weight parsing network provides basic learning samples and knowledge sources through the first training data and the two-dimensional labels of scene categories, and performs joint learning and balancing of each loss term to achieve modeling and optimization of subdivided scenes.

[0059] In the joint training process of the weight parsing network and the PDP model to be trained, the PDP model is trained based on the first training data and two-dimensional labels of scene categories, so as to dynamically integrate the PDP output of the PDP model through the loss term. Specifically, by generating adaptive loss term weights through the weight parsing network, the PDP output can be optimized in terms of security loss, user experience comfort, and operational efficiency, including the security loss, comfort loss, and efficiency loss included in the loss term.

[0060] Based on this, the various loss terms are weighted and fused to form a weighted loss, which guides the weight parsing network and the PDP model to adjust parameters during backpropagation, thereby achieving collaborative optimization and overall performance improvement of the weight parsing network and the PDP model.

[0061] Step 103: Based on the loss term and the weights of the matched loss term, the trained decision planning model is obtained.

[0062] The aforementioned loss items may include, but are not limited to, at least one of safety loss, comfort loss, imitation loss, and efficiency loss. Of course, the loss items here are merely examples; any other loss items related to autonomous driving, planning and control, reinforcement learning, and end-to-end imitation learning fall within the protection scope of this application's embodiments, and will not be listed here individually. Furthermore, the above example of 10 digital tags is merely for illustration; the number of digital tags is not limited here and depends on the specific sub-scenario.

[0063] Correspondingly, the weight parsing network can automatically and dynamically generate loss term weights that are adapted to the current scenario category during the training process of the decision planning model to be trained. For example, the loss term weights can include, but are not limited to, the respective weights of safety loss, comfort loss, and efficiency loss to represent the relative weights of safety loss, comfort loss, and efficiency loss, indicating the specific weights that are more emphasized among these loss term weights.

[0064] The aforementioned safety losses are analyzed and assessed quantitatively using at least one indicator, such as the distance between the vehicle and the obstacle, relative speed, and time to collision (TTC), to determine the losses caused by collision risk, violation of traffic rules, or being in an unsafe driving state.

[0065] The aforementioned comfort loss is achieved by monitoring and evaluating at least one parameter, such as longitudinal acceleration, lateral acceleration, and jerk, to ensure the smoothness of vehicle operation and the comfort of passengers.

[0066] The aforementioned imitation loss is calculated by comparing the deviations between the autonomous vehicle's actual driving behavior and the human driver's demonstration behavior in the same or similar scenarios by comparing the autonomous vehicle's trajectory, speed curve, steering angle, accelerator and brake pedal operation with human demonstration data.

[0067] The aforementioned efficiency loss is quantified by comparing and analyzing the driving time, average speed, energy consumption indicators with at least one corresponding indicator under the optimal or desired state. This results in situations such as increased driving time, increased fuel consumption, or increased electricity consumption due to at least one of the following: unreasonable route planning, improper speed control, or failure to effectively utilize road resources.

[0068] Based on this, by dynamically and adaptively determining the weights according to the specific scenario categories, a balanced optimization of multiple objectives such as safety, comfort, efficiency, and mimicking human driving behavior can be achieved.

[0069] In practical applications, scene categories are divided into dimensions based on the target object. For example, there are dimensions for car interaction, human interaction, and static roadside interaction. First, the dimension of the target object is determined, and then further subdivided based on the specific interactive behaviors or basic scenarios within that dimension. This allows for rapid matching of scene categories.

[0070] In related technologies, whether it's fixed weight setting, posterior weight adjustment, or association rules derived from human experience based on scene labels, all rely to varying degrees on the human experience of engineers. This approach is not only inefficient and highly subjective, but also struggles to cover the massive and complex sub-scenarios in the field of autonomous driving. It cannot achieve adaptive dynamic optimization of loss term weights, resulting in limited generalization ability and difficulty in adapting to the decision-making needs of all scenarios.

[0071] Compared to related technologies that rely on engineers' human experience, which are inefficient, subjective, and unable to cover the vast and complex sub-scenarios in the autonomous driving field, and cannot achieve adaptive dynamic optimization of loss term weights, resulting in limited generalization ability and difficulty in adapting to decision-making needs across all scenarios, the weight parsing network in this application quickly matches the corresponding loss term weights based on the input two-dimensional labels. This allows the loss term weights to closely match the dynamic characteristics of the current scenario, which is beneficial for training the decision planning model, thereby improving the accuracy and reliability of scene recognition. It also avoids the limitations of traditional fixed-weight training methods in complex and ever-changing scenarios, thus improving the adaptability and prediction accuracy of the decision planning model in different scenarios. Furthermore, it does not rely on engineers' human experience, is free of subjectivity, is highly efficient, covers a vast and complex sub-scenarios in the autonomous driving field, achieves adaptive dynamic optimization of loss term weights, has strong generalization ability, and adapts to decision-making needs across all scenarios.

[0072] In this paper, at least one of the following optional methods can be used to output the loss term weights that dynamically match the decision result: In the first alternative approach, weight smoothing regularization is introduced during the joint training phase of the decision planning model to be trained and the weight parsing network. By determining the perceptual feature similarity between samples within the training batch, constraints are imposed on the weight differences of similar samples to generate differentiated weights that adapt to different environmental details of the same scene category, so as to obtain loss term weights that dynamically match the decision result.

[0073] Specifically, if two samples have the same scene category and similar perceptual details (e.g., pedestrian distances of 3m and 3.2m), the regularization term will force the weights output by the weight parsing network to be consistent, avoiding the fragmentation of the weight space caused by generating independent weights for each sample.

[0074] For samples with significant differences in perceived details, such as pedestrian distances of 1m and 5m, the weight parsing network outputs differentiated weights to adapt to different decision-making needs. In this way, it ensures that the weights adapt to perceived details, and also ensures the continuity and smoothness of the weight space through regularization constraints, effectively improving the convergence and generalization ability of the model.

[0075] Combination Figure 2 and Figure 3 In the second optional approach shown, the weight parsing network learns the scene category, dynamic interaction information, and loss term weights, and outputs loss term weights that dynamically match the decision result. This may include, but is not limited to, the following: the weight parsing network deeply fuses the two-dimensional label of the scene category with the perceptual information to generate differentiated weights that adapt to different environmental details of the same scene category, so as to obtain loss term weights that dynamically match the decision result.

[0076] The mapping relationship between scene categories and loss term weights in related technologies is determined solely by the two-dimensional label of the scene category, without integrating real-time perception information. This makes it difficult to effectively distinguish the weights corresponding to different perception details in the same type of scene (such as pedestrian density versus no pedestrians in a straight-through scene on an urban road, or large vehicles in an adjacent lane versus no vehicles in an adjacent lane). The weight generation is too coarse-grained and cannot match the dynamic decision-making needs of the actual environment, which may lead to the risk of decision preferences becoming disconnected from the real-time environment.

[0077] Compared to related technologies where weights are difficult to differentiate effectively and the weight generation is too coarse-grained, failing to match the dynamic decision-making needs of the actual environment and easily leading to the risk of decision preferences being out of sync with the real-time environment, this application embodiment generates differentiated weights adapted to different environmental details of the same scene category to obtain loss item weights that dynamically match the decision results. This enables the weight parsing network to capture perceptual detail differences in the same scene: for example, in a straight-through scenario on an urban road, the network can distinguish between dense pedestrian traffic and no pedestrian traffic, large vehicles in adjacent lanes and no vehicles in adjacent lanes, etc., through changes in edge weights, and thus generate differentiated loss item weights (such as automatically increasing the safety loss item weight when pedestrian traffic is dense).

[0078] Thus, the adaptive generation method of loss term weights breaks the limitations of traditional coarse-grained weights, enabling weight allocation to match the dynamic decision-making needs of the actual environment, and fundamentally avoiding the risk of decision preferences becoming disconnected from the real-time environment.

[0079] As an optional embodiment of this application, the weight parsing network includes a graph neural network. Thus, the weight parsing network structure employs a graph neural network (GNN), defining the two-dimensional labels, static elements, and dynamic elements of the scene category as graph nodes. Based on the matching degree between graph nodes and the scene, as well as the spatial and / or motion conflict relationships between graph nodes, dynamic edge weights are constructed. The deep aggregation of the associated features of the basic scene category and perceptual detail information is achieved through the message passing mechanism of graph convolution. Finally, a loss term weight matrix adapted to the current scene is output through a fully connected layer.

[0080] In order to dynamically match the loss term weights with the decision result, at least one of the following optional embodiments can be adopted: the two-dimensional labels of the scene category are deeply fused with the perceived information to generate differentiated weights that adapt to different environmental details of the same scene category, which serve as the loss term weights dynamically matched with the decision result: In a first optional embodiment, steps 1 through 3 may be included, but are not limited to: The first step involves constructing an association graph from the two-dimensional labels, static elements, and dynamic elements of the scene categories in the perceived information. The graph nodes of the association graph include scene label nodes representing scene category attributes, as well as static element nodes and dynamic element nodes. The edge weights of the association graph are constructed based on the matching degree between graph nodes and the scene, and the spatial and / or motion conflict relationships between graph nodes. The elements include the static elements and the dynamic elements.

[0081] Specifically, graph nodes include scene nodes that represent scene category attributes, static element nodes that represent static environmental features such as lane lines and curbs, and dynamic element nodes that represent dynamic traffic participants such as vehicles and pedestrians. The graph edge weights are not fixed but are dynamically updated based on real-time perception information. For example, the edge weights between pedestrian nodes and straight-through scene nodes on urban roads are dynamically adjusted according to the distance between pedestrians and vehicles and their relative speed. The closer the distance and the higher the risk of conflict, the greater the edge weight.

[0082] like Figure 3 As shown, the weighted parsing network has two inputs: one is a two-dimensional label for the scene category; the other is perceptual information derived from the scene recognition model, which is real-time perceptual information. Convert the two-dimensional labels of scene categories, such as label 1 for "straight ahead on a city road" and "VRU crossing", into one-hot encoded vectors. .in, This represents the total number of scene categories, used to characterize the macro-level decision-making preference for the current scene category. It represents the real number field, meaning that every element in the matrix is ​​a real number. The shape of the real number field (number of rows × number of columns) is represented in this article. The superscripts in the upper right corner all indicate the shape of the real number field (number of rows × number of columns), which will not be elaborated here.

[0083] The aforementioned perceptual information includes the following static element feature tensor, namely: and dynamic element feature tensor, i.e. .

[0084] in, This represents the number of static elements. This represents the number of sampling points for static elements. This refers to the semantic feature dimension of static elements, such as the curvature and position of lane lines; For the number of dynamic elements, For the temporal feature dimension of dynamic elements, This refers to the motion characteristic dimensions of dynamic elements, such as the position, speed, and direction of movement of pedestrians.

[0085] The second step involves dynamically updating the edge weights to obtain the perceived changes in different environmental details.

[0086] The third step is to integrate the global features of scene category attributes and environmental details of the same scene to generate differentiated weights.

[0087] Compared to current related technologies that only use scene identifiers as input parameters for scene and weight mapping, and fail to consider the fusion of perceptual information and scene categories, resulting in coarse-grained weight generation and decision preferences easily becoming disconnected from the real-time environment, this application embodiment incorporates a dynamic association graph modeling module within the weight parsing network. This overcomes the limitations of related technologies that only use scene identifiers as input parameters, achieving deep fusion of scene categories and perceptual information, and realizing the association representation of scene categories and perceptual information. This generates fine-grained weights that adapt to different environmental details within the same scene. Thus, scene nodes can receive dynamic feature feedback from all perceptual element nodes, and perceptual element nodes can also obtain macro-level preference guidance from scene nodes, ultimately outputting global features that fuse macro-level scene attributes and micro-level perceptual details, supporting fine-grained weight generation.

[0088] Furthermore, in this embodiment, a relational graph is constructed from two-dimensional labels of scene categories, static elements, and dynamic elements. By dynamically updating edge weights, subtle changes in perception are captured. The resulting global features, fused with macroscopic scene attributes and microscopic environmental details, are aggregated through a GNN neighborhood and used to generate differentiated weights. For example, the weight of safety loss items is automatically increased when pedestrian traffic is dense. This ensures that the allocation of loss item weights matches the dynamic decision-making needs of the actual environment.

[0089] As an optional embodiment of this application, the third step described above may include, but is not limited to: aggregating the dynamic feature feedback and preference guidance of all nodes through neighborhood features, outputting global features that fuse the scene category attributes and environmental details of the same scene, and generating differentiated weights.

[0090] Among them, the dynamic feature feedback and preference guidance of all nodes are used to represent the real-time feature information received by scene nodes from static element nodes and dynamic element nodes, as well as the direction and degree of influence of scene nodes on each element node based on the scene category attributes they represent.

[0091] For example, in a straight-ahead scenario on an urban road, the macro-level preference guidance of scene nodes may focus more on the driving status of the vehicle ahead (dynamic element node) and the integrity of the lane line (static element node). In this case, the dynamic features fed back to the scene node by the vehicle ahead node and the lane line node, such as vehicle speed, acceleration, and lane line curvature, are given higher attention weight.

[0092] The aforementioned scene nodes transmit stronger preference guidance to these other nodes, prompting them to play a greater role in feature aggregation. Thus, this two-way dynamic feature feedback and preference guidance allows neighborhood features to effectively integrate the macroscopic attributes of the scene with perceived microscopic environmental details. This ensures that the generated differentiated weights not only match the scene category but also adapt to the specific environmental conditions of the current scene. For example, detailed information such as traffic flow, road curvature, and obstacle distribution facilitates the dynamic matching of subsequent loss term weights.

[0093] Specifically, the aforementioned neighborhood features can include neighborhood features obtained through the GNN. Thus, the GNN weight parsing network generates weights based on a combination of basic scene category and perceptual detail information. Even for the same scene type, such as driving straight on an urban road and a VRU crossing, if the perceptual detail information differs significantly, such as different pedestrian distances or different neighboring vehicle states, the weight parsing network will output different loss term weights.

[0094] Specifically, the above graph node definition is as follows: the two-dimensional labels of scene categories, static elements, and dynamic elements are respectively treated as graph nodes, and the node set is:

[0095] The feature of each node is its corresponding input vector.

[0096] The edge weights in the association graph in this paper are not fixed values, but are dynamically updated based on the matching degree between graph nodes and the scene, the conflict risk between nodes, and other factors. Taking pedestrian nodes as an example... Scene label nodes used to represent scene category attributes edge weight For example, the calculation method is as follows:

[0097] in, These are the feature vectors of pedestrian nodes, and we need to work on the dynamic element feature tensor. This involves fusing the temporal and motion features of a single pedestrian along a temporal dimension, achieved using temporal average pooling. The specific formula is as follows:

[0098] After fusion, the result is It includes time-series aggregated information such as distance / relative speed to the vehicle, realizing the time-series dimension. Effective integration; in, These are learnable parameters; σ The sigmoid activation function has an output range of [0, 1]. A larger value indicates a stronger influence of the pedestrian on scene decisions. Similarly, for static element nodes such as lane lines... eigenvectors of static element nodes The static element feature tensor needs to be processed first. That is, the sampling points and semantic features of a single lane line are fused together using spatial sampling point dimensions, and the fused result is... Then it participates in the edge weight calculation.

[0099] The above graph feature aggregation is implemented as follows: ① The message passing and aggregation of node features are achieved through the graph convolutional layers of GNN. The node feature update formula for the l-th layer is:

[0100] in, It is a node The set of neighboring nodes, These are the learnable parameters of the l-th layer; ReLU is the activation function.

[0101] ② After 2-3 layers of graph convolution, global pooling is performed on the features of all nodes to obtain a global feature vector that integrates the correlation information of basic scene category and perceptual detail information. D is the feature dimension.

[0102] ③ Adaptive loss weight generation: This involves generating global feature vectors... Input a fully connected layer, output a loss weight matrix that adapts to the current scene and perceived details:

[0103] in, , These are learnable parameters. The Softmax function guarantees that the sum of the weights is 1, and the output... The corresponding weights for the loss items are: imitation loss, safety loss, comfort loss, and efficiency loss.

[0104] Example: In a scenario where a VRU traverses a city road with the digital label 1, if the perceived distance between the pedestrian and the vehicle is less than 5 meters, the edge weights from the pedestrian node to the scene node in the GNN graph will increase, and the generated weight matrix may be as follows: The proportion of safety losses increases; if there are "no pedestrians" in the same scenario, the weight may be [missing information]. The weighting of efficiency loss has increased.

[0105] ④ Joint training of models: Continue as follows Figure 3 As shown, the weights output by the weight parsing network are combined with the loss term of the PDP model, and a weight smoothing regularization constraint is introduced to achieve synergistic optimization of adaptive differentiation and convergence stability.

[0106] After the above PDP model outputs the decision results, the following imitation loss is determined. (Measuring the difference between human and non-human driver decision-making), safety losses (Measuring collision risk) and loss of comfort (Measures acceleration fluctuations) and efficiency losses Four types of losses, including (measuring traffic speed).

[0107] Based on the above four types of losses, the weighted total loss is obtained as follows:

[0108] in, This is a weight smoothing regularization term used to constrain samples with similar perceived details, ensuring that their weights are similar and avoiding excessive weight dispersion. This is the regularization strength coefficient, which can be adaptively adjusted based on training convergence. For example, a smaller value can be set in the early stages of training to ensure differentiation, while a larger value can be added later to improve smoothness. (Regularization term) The weighted distance loss is calculated using perceptual feature similarity weights, and the specific formula is as follows:

[0109] in, This represents the number of samples in the current training batch. The perceptual feature similarity between sample i and sample j is calculated using cosine similarity, i.e.:

[0110] in, The closer to 1, the more similar the scene categories and perceptual details of the two samples are; The squared L2 norm measures the difference between the two sets of weights. Thus, samples with similar perceived details ( The weight differences are strictly constrained (resulting in increased loss), forcing the network to generate a continuous, smooth weight space, rather than generating independent weights for each sample.

[0111] On this basis, continue as Figure 3 As shown, backpropagation optimization is performed in the following way: based on the total loss with regularization. Simultaneously, the parameters of the weight parsing network and the PDP model are updated, and the Adam optimizer is used for iterative training until the loss converges. At this point, the network learns the mapping relationship between scene category and perceptual detail information and the loss weights. This ensures that the weights of different perceptual details in the same scene are differentiated, while avoiding the model non-convergence problem caused by excessive weight fragmentation.

[0112] Compared to related technologies that rely on human experience through scene and weight mapping schemes and lack data-driven adaptive capabilities, this application's embodiments utilize a data-driven dual-input weight parsing network to replace the traditional model of manually setting weights, defining scene recognition, and associating loss term weights. On one hand, this eliminates the need for subjective engineer intervention, significantly improving weight generation efficiency and avoiding the limitations of human experience. On the other hand, the network can autonomously learn basic scene categories, perceive detailed information, and associate loss term weights through massive training data, enabling it to cover various complex and segmented scenes, achieve adaptive dynamic optimization of loss term weights, improve the generalization ability of the solution, and ensure adaptability to all scenarios.

[0113] Thus, the data-driven adaptive weight parsing network, employing dual inputs and graph-structured feature aggregation, replaces the traditional manual intervention model. At the input level, two types of information are introduced simultaneously: first, the scene category labeled during the data acquisition process, providing macroscopic prior information about the scene; and second, real-time perception information from the same source as the PDP model input, providing microscopic details of the environment.

[0114] Of course, the number of loss terms in this embodiment is related to the loss required to adapt to the scene category. Specifically, the combination and number of loss terms are flexibly adjusted according to the characteristics of different scene categories. In this way, the dynamic adjustment of the number and combination of loss terms enables the weight parsing network to be optimized for the needs of the current scene, further improving the adaptability and decision quality of the decision planning model in different scenarios.

[0115] In a second optional embodiment, the number of loss item weights dynamically matched with the decision result is determined in real time based on the scenario category. When there are multiple loss item weights dynamically matched with the decision result, the loss item includes multiple loss items.

[0116] In response, the weight parsing network learns the two-dimensional labels of the scene category and the perceptual information, and outputs loss term weights that are dynamically matched with the decision result. This may include, but is not limited to, the following: the weight parsing network learns the two-dimensional labels of the scene category and the perceptual information, and outputs loss term weights that are dynamically matched with the decision result. Accordingly, obtaining the trained decision planning model includes: determining the loss value of each loss term between the decision result and the actual result according to each loss term and the weight of each loss term dynamically matched with the decision result, obtaining the total loss value, and stopping the iteration when the total loss value converges to a stable state, thereby obtaining the trained decision planning model.

[0117] Furthermore, based on the convergence state of the total loss value obtained from the loss values ​​of each loss in the current iteration round and the previous round, it is determined whether to continue iterative training. When the total loss value converges to a stable state, the iteration is stopped, and the trained decision planning model is obtained as the decision planning model.

[0118] In this embodiment of the application, by introducing multiple loss terms and dynamically matching the weights of each loss term, the decision planning model can adaptively adjust the contribution of each loss term to the total loss value according to different scenario categories and dynamic interaction information during the training process.

[0119] For example, in complex traffic scenarios involving the interaction of multiple vehicles, the weights of loss terms related to vehicle collision risk can be dynamically increased, thereby enabling the decision planning model to pay more attention to the optimization of such key safety indicators during training.

[0120] For example, in relatively simple straight-line driving scenarios, the weights of loss terms related to the smoothness of the driving trajectory may be given higher priority in order to improve the comfort and rationality of the output trajectory of the decision planning model.

[0121] This breaks the limitations of the traditional fixed-weight training model, enabling the decision-making and planning model to learn decisions in different scenarios, thereby effectively improving the generalization ability and decision accuracy of the decision-making and planning model in various practical application scenarios.

[0122] In a third optional embodiment, the number of loss terms is determined in real time according to the scenario category. When the number of loss term weights dynamically matched with the decision result is 1, the corresponding loss term includes one loss term. The process of obtaining a trained decision planning model based on the loss term and the matched loss term weights may include, but is not limited to: determining the loss value of the loss term between the decision result and the actual result according to one loss term and the loss term weight dynamically matched with the decision result, stopping the iteration when the loss value converges to a stable state, and obtaining the trained decision planning model.

[0123] Furthermore, based on the convergence state of the loss value between the current iteration round and the previous round, it is determined whether to continue iterative training. When the loss value converges to a stable state, the iteration is stopped, and the trained decision planning model is obtained as the decision planning model.

[0124] In this embodiment of the application, when the number of loss items is 1, it indicates that only one loss item is used at this time, which can better reflect the loss item with the strongest demand among multiple loss items, so as to meet the needs of the current scenario.

[0125] For example, in scenarios where only a single metric needs to be considered, such as low-speed cruising in a closed park, it might only be necessary to set a loss term related to obstacle avoidance. Through dynamic weight adjustments, the model can be optimized to ensure that decision-making and planning in this scenario meets the most critical safety or efficiency objectives. In this way, the combination of a single loss term and dynamic weights simplifies model training complexity and allows for effective adaptation to scenario changes through adaptive weight adjustments, thereby improving the efficiency and reliability of the decision-making and planning model in scenarios with a single loss requirement.

[0126] Combination Figure 2 and Figure 3 As shown, as an optional embodiment of this application, the above step 110 may include, but is not limited to, the following steps 1 and 2: Step 1, based on the two-dimensional labels of scene categories in the first training data and the encoded features of the perception information, obtain fusion features that include environmental information and scene context.

[0127] Step 1 above may include, but is not limited to, the following two steps: First, converting the two-dimensional labels of the scene categories in the first training data into one-hot encoded features. Second, weighted fusing the one-hot encoded features with the encoded features of the perceptual information to obtain fused features that include environmental information and scene context.

[0128] In response, the input to the decision planning model contains two types of information, which are encoded and fused into a unified feature: the perception information includes the feature tensors of collected static elements (such as lane lines, curbs, etc.) and the feature tensors of dynamic elements (such as vehicles, pedestrians, etc.), which serve as the environmental information input to the decision planning model; the two-dimensional label fusion of scene categories: the combined labels of the labeled basic scene and dynamic interaction, such as the numerical labels 1 and 10 in Table 1 above, are converted into one-hot encoded features, which are then weighted and fused with the encoded features of the perception information, enabling the decision planning model to directly obtain the contextual information of the current scene.

[0129] Step 2: Input the fused features into the decision planning model and weight parsing network to be trained, respectively.

[0130] The weight parsing network is used to dynamically parse the weight values ​​of each loss term based on the fusion features, so that different loss terms can be weighted differently during the subsequent model training process.

[0131] For example, when the fused features contain a large amount of complex traffic participant interaction information, the weight parsing network may assign higher weights to loss terms related to vehicle following, lane changing, etc.

[0132] For example, when the fused features indicate that the current road is structured and the traffic flow is stable, the weight of the loss term related to driving efficiency may be appropriately increased, thereby making the model training more targeted and adaptable to different scenarios.

[0133] Specifically, the weight parsing network dynamically generates weight parameters corresponding to each loss term based on environmental information (such as road type, traffic flow, weather conditions, etc.) and scene context (such as whether the current driving task is high-speed driving or low-speed congestion following other vehicles) contained in the fused features.

[0134] In this embodiment, the weight parsing network is used to perform deep analysis of the fused features to explore the importance of each loss term in different scenarios. This allows the decision planning model to be trained to adaptively adjust the importance of each loss term based on the specific scenario environment and context information reflected by the current input fused features. As a result, loss term weights that are adapted to the current scenario can be automatically generated, realizing scenario-based dynamic adjustment of loss term weights.

[0135] The PDP model in related technologies relies solely on perceived information to autonomously determine scene attributes, making it difficult to efficiently distinguish the semantic differences of the same perceived information in different scenarios. For example, when perceived information indicates that a vehicle in front in another lane has activated its turn signal, the PDP model cannot quickly determine whether the behavior is "normal lane change on a highway" (the vehicle has options for acceleration / non-deceleration driving—lane change prohibited, and deceleration to yield—lane change permitted) or "lane merging or lane change based on navigation information" (requiring advance deceleration and leaving room for avoidance). This can easily lead to confusion in decision-making logic, resulting in insufficient understanding of complex scenarios by the trained model.

[0136] To address the technical problem of insufficient understanding of complex scenarios by the PDP models in the aforementioned related technologies, this application provides a vehicle decision-making and planning method. A scene recognition model trained using training data and two-dimensional labels representing the basic scenes and their corresponding dynamic interaction information can identify the basic scenes and their dynamic interaction information. The trained decision-making and planning model can then output planning and control commands adapted to the current scene category. This quickly distinguishes the interactive actions of the current scene, thereby improving the understanding of complex scenarios and enabling the generation of planning and control commands adapted to the current scene category, which is beneficial for more precise control.

[0137] Figure 4 The diagram shown is a flowchart of the vehicle decision-making and planning method provided in an embodiment of this application.

[0138] like Figure 4 As shown, the vehicle decision-making and planning may include, but is not limited to, the following steps 201 to 203: Step 201: Obtain the current perception information of the current scene collected by the actual vehicle's sensors.

[0139] Step 202: Identify the current scene category based on the current perceived information.

[0140] Step 203: Input the current scene category and current perception information into the trained decision planning model so that the trained decision planning model can output planning control instructions that are adapted to the current scene category according to the decision preference of the current scene category; the trained decision planning model is a decision planning model trained using the decision planning model training method described above.

[0141] The trained decision-making and planning model, through learning from a large amount of scenario data, can develop corresponding decision logic and behavioral patterns for different scenario categories. For example, in congested urban road scenarios, decision preferences may focus more on smooth vehicle driving and following safety. In highway scenarios, however, the emphasis is on driving efficiency and lane-keeping stability. When fused features are input into the decision-making and planning model, the model comprehensively analyzes environmental information and scenario context based on the decision preferences corresponding to the current scenario category, thereby outputting specific planning and control commands, including vehicle acceleration, deceleration, steering, and lane changes, to achieve adaptive and intelligent vehicle control in different scenarios.

[0142] To address this, the offline-trained scene recognition network and weight parsing network are deployed on a real-vehicle computing platform through integrated online inference of scene recognition and decision planning models, and run in real time in collaboration with the decision planning model.

[0143] The decision-making and planning models in this paper may include, but are not limited to, the PDP model. The technical approach of the PDP model has also gradually shifted from the traditional architecture containing rules and models to a more generalizable end-to-end (E2E) network architecture as the functional hierarchy evolves.

[0144] In this embodiment, after identifying the current scene category through a scene recognition model, feature fusion is performed, and a planning and control command adapted to the current scene is obtained through reasoning by a decision planning and planning model. Thus, the sequential process of the scene recognition model and the decision planning and planning model enables online collaboration between perceived information and scene information, allowing the decision planning and planning model to adapt to the decision-making needs of different sub-scenarios in real time.

[0145] Step 202 above can be performed using at least one of the following optional methods to identify the current scene category based on the current perceived information: In the first alternative approach, historical scene data collected by vehicle sensors can be pre-labeled by humans to categorize scenes. When identifying the current scene category, the current perceived information is compared with the labeled historical scene data. If the similarity between the features of the current perceived information and the features of a certain historical scene category exceeds a preset threshold (e.g., 90%), then that historical scene category is determined as the current scene category. This approach relies on the accuracy and coverage of manual labeling and is suitable for situations where scene categories are relatively fixed and their features are clearly defined.

[0146] Figure 5 The diagram shown is a flowchart illustrating step 202 of the vehicle decision-making and planning method provided in this embodiment of the application.

[0147] like Figure 5 In the second alternative shown, step 202 may include, but is not limited to, steps 110 to 120: Step 110: Input the current perception information into the scene recognition model to output the scene category probability distribution; the scene recognition model is trained based on the training data and the two-dimensional labels of each basic scene and its corresponding dynamic interaction information corresponding to the scene category.

[0148] The scene category probability distribution represents the probability value of the current perceived information belonging to different scene categories. For example, when a vehicle in another lane activates its turn signal, the scene recognition model outputs the probability distribution for each scene category, considering both the basic scene of the perceived information (such as an intersection, ramp, construction zone, lane merging section on a regular highway, or navigation-based lane change) and dynamic interaction information (such as the relative speed, distance, and relative position of the vehicle and other vehicles). For instance, the probability of the vehicle going straight and the other vehicle turning right at an intersection, an ramp, a construction zone, a lane merging / lane change, or navigation-based lane merging / lane change is given. These probability values ​​quantify the degree of matching between the current perceived information and each scene category, facilitating accurate subsequent scene category determination.

[0149] The scene recognition model mentioned above that inputs current perception information refers to a trained scene recognition model, which enables online identification of the specific scene category in the current scene, i.e., the current scene category.

[0150] Example 1: The scene recognition model is trained based on training data and labeled basic scenes and their corresponding dynamic interaction information, along with two-dimensional labels for scene categories. Specifically, A scene classification network is constructed based on a deep learning model to serve as the scene recognition model. For example, a Convolutional Neural Network (CNN) or a Recurrent Neural Network (RNN) can be used. Taking the current perceived information as input, the network extracts features from multiple layers and the classifier outputs to directly determine the current scene category. When training this scene classification network, a large amount of historical perceived information with scene category labels is used as training samples. The network parameters are continuously adjusted through backpropagation to improve the accuracy of scene classification.

[0151] Step 120: Based on the probability distribution of scene categories, determine the current scene category that contains the basic scene and its corresponding dynamic interaction information.

[0152] In this embodiment, by inputting the current perceived information into a pre-trained scene recognition model, the model can effectively output scene category probability distributions through dual learning of basic scene and dynamic interaction information. Based on these probability distributions, the specific scene category of the current scene can be accurately determined, thus avoiding the decision-making logic confusion caused by relying solely on single perceived information in related technologies and improving the recognition accuracy of complex scenes. Simultaneously, the scene recognition model automatically identifies the current scene category to improve the accuracy of scene classification.

[0153] Combination Figure 5 As shown, step 120 above can employ at least one of the following optional embodiments to determine the current scene category containing the basic scene and its corresponding dynamic interaction information: In a first optional embodiment, the scene category with the highest probability in the scene category probability distribution is taken as the current scene category.

[0154] For example, if the scene recognition model outputs a scene category probability distribution for a given current perceived information, and the probability of a vehicle going straight and another vehicle turning right at an intersection is 0.65, the probability of a vehicle going straight and another vehicle turning right at a ramp is 0.08, the probability of a vehicle going straight and another vehicle turning right at a construction site is 0.02, the probability of a vehicle going straight and another vehicle turning right at a lane merging / changing scenario is 0.1, the probability of a vehicle going straight and another vehicle turning right based on navigation information is 0.03, the probability of a lane merging / changing scenario is 0.12, and the probability of a lane changing scenario based on navigation information is 0.03, then the sub-scene with the highest probability, "a vehicle going straight and another vehicle turning right at an intersection," is directly identified as the current scene category. A two-dimensional label for the scene category serves as the scene category identifier.

[0155] In this embodiment, the results of probability statistics can reflect the matching degree between the current perceived information and the scene category to the greatest extent. By selecting the scene category with the highest probability value in the scene category probability distribution as the current scene category, the specific category of the current scene can be determined intuitively and efficiently, thereby ensuring that the current scene category has high reliability and accuracy, and facilitating subsequent vehicle control based on the scene category.

[0156] In a second optional embodiment, if there are multiple scene categories with the highest probability in the scene category probability distribution, the scene category with the highest probability is taken as the current scene category.

[0157] For example, the probability of a car going straight and another car turning right at an intersection is 0.45, the probability of a car going straight and another car turning right at a ramp is 0.45, and the probability of other scenarios is all below 0.45.

[0158] In this embodiment, when two or more scene categories with the same and highest probability values ​​appear in the scene category probability distribution output by the scene recognition model, any one of these highest-probability scene categories can be randomly selected as the current scene category. This effectively addresses the special case of a flat probability distribution, avoiding decision-making stagnation caused by multiple scene categories having the same highest probability. By allowing arbitrary selection, the continuity and real-time nature of the scene category determination process are ensured, improving the smooth operation of the vehicle control process. Furthermore, since these selected scene categories themselves have the highest probability, even random selection largely guarantees the rationality of the chosen scene category.

[0159] In a third optional embodiment, if there are multiple scene categories with the highest probability in the scene category probability distribution, the scene category with the highest security weight among the multiple scene categories with the highest probability is taken as the current scene category.

[0160] Safety weights are pre-set for different scenario categories. These safety weights are used to quantify the safety risk level of vehicle driving under different scenario categories. The higher the safety weight, the higher the requirements for safe vehicle driving under the corresponding scenario category, and the more conservative or more detailed control strategies need to be adopted.

[0161] In this embodiment, when multiple scene categories with the same and highest probability values ​​appear in the scene category probability distribution output by the scene recognition model, a preset safety weight table is invoked. The safety weights corresponding to these highest-probability scene categories are compared, and the scene category with the highest safety weight is selected as the current scene category. This facilitates decision-making from a safety perspective, prioritizing vehicle driving safety in complex or high-risk scenarios. By proactively selecting scene categories with higher safety weights, the final control is more safety-oriented, thereby further reducing potential safety hazards and improving vehicle driving reliability.

[0162] In a fourth optional embodiment, the scene category corresponding to the scene category probability greater than the confidence threshold is taken as the current scene category.

[0163] The aforementioned confidence threshold represents the minimum level of confidence that the probability of a scene category should reach. Its specific value can be preset according to the needs and accuracy requirements of the actual application scenario. For example, if the confidence threshold is set to 0.7, then when the probability value of a certain scene category output by the scene recognition model is greater than or equal to 0.7, the scene category is considered to have sufficient confidence and can be directly identified as the current scene category.

[0164] In this embodiment of the application, by setting a reasonable confidence threshold, scene categories with low probability and insufficient confidence can be effectively filtered out, avoiding interference with subsequent vehicle control decisions due to ambiguous or uncertain recognition results. This enables the determination of scene categories quickly and directly while ensuring recognition accuracy, thereby improving the clarity and reliability of vehicle control strategies.

[0165] Of course, the confidence threshold mentioned above can be flexibly set and adjusted according to actual application needs. For example, in areas or time periods with extremely high requirements for driving safety, the confidence threshold can be set to a higher value to ensure that when the scene recognition model has a very high degree of certainty in judging a certain scene category, it is taken as the current scene category, thereby avoiding potential risks caused by the uncertainty of the model's judgment.

[0166] In some regular driving scenarios with higher real-time requirements and a certain margin of error, the confidence threshold can be appropriately reduced to ensure the timeliness and continuity of scenario category recognition.

[0167] In a fifth optional embodiment, if the scene category is determined to be a scene category that is less than a confidence threshold, the scene category that is less than the confidence threshold is taken as the current scene category.

[0168] If no scene category reaches the aforementioned confidence threshold, a safety fallback decision can be added. This means that when the confidence level for all scene categories is below the set threshold, a pre-defined conservative control strategy is adopted by default, such as reducing vehicle speed, increasing safe following distance, and enhancing environmental perception frequency. This conservative decision-making approach maximizes vehicle safety when scene recognition results are uncertain, avoiding dangers caused by blindly relying on low-confidence recognition results.

[0169] In this embodiment, a safety fallback decision can be added when no scene category reaches the aforementioned confidence threshold. This not only ensures that vehicle control decisions can be quickly adapted based on the scene category when the scene recognition result is clear and the confidence level is sufficient, but also avoids the risk of decision interruption or loss of control due to missing scene information by using the low-confidence scene category as the current scene category in special cases where the confidence level of a scene category is lower than the set threshold.

[0170] Furthermore, compared to the end-to-end real-time response of the current scenario category input to the subsequent decision planning model adaptation in related technologies, this allows the decision planning model to call the scenario-based decision trained offline, while avoiding the risk of scenario recognition failure through safety fallback decision, thus comprehensively improving the decision adaptability and safety of real vehicle operation.

[0171] The above method may also include, but is not limited to, the following steps 210 to 220, to train the scene recognition model: Step 210: Obtain the second training data; the second training data includes: perception information collected by at least one sensor of the actual vehicle, and is used to label each basic scene with a two-dimensional label corresponding to the scene category of the dynamic interaction information.

[0172] The second training data mentioned above is multimodal perception information, used to represent basic scene and dynamic interaction information, which can comprehensively reflect the detailed traffic scene in which the vehicle is located, also known as segmented scene. The second training data uses sensor data collected from real vehicles, and is labeled with two-dimensional labels for basic scene and dynamic interaction.

[0173] Next, for each training data point, its corresponding basic scene category is manually labeled, along with detailed annotations of the dynamic interaction information within that basic scene. Then, the basic scene category and the dynamic interaction information are combined to form a two-dimensional label; that is, the label for each training sample consists of a basic scene label and a dynamic interaction information label.

[0174] The two-dimensional labels mentioned above not only clearly define the categories of each basic scenario, but also correspond to the dynamic interactive information under each basic scenario.

[0175] Next, the aforementioned two-dimensional label refers to a label obtained by combining and annotating basic scene and dynamic interaction information on images in the perceived information. This two-dimensional label represents the combination relationship between the basic scene category and the dynamic interaction information; that is, a specific traffic scene instance can be uniquely identified through the two-dimensional label. For example, the two-dimensional label includes, but is not limited to, a digital label for an image, such as one-hot encoding. The two-dimensional dimension of the two-dimensional label represents both the dimension of the basic scene and the dimension of the dynamic interaction.

[0176] For example, when the basic scene category is detailed road segment information of urban roads, and the dynamic interaction information is vehicle-pedestrian interaction, or interaction between the vehicle and other vehicles, the two-dimensional label can easily correspond to the subdivided scene of vehicle-pedestrian interaction in the urban road environment. In this way, the second training data can characterize the features of different subdivided scenes, which facilitates the training of the scene recognition model and helps the scene recognition model learn the inherent relationship between the basic scene and dynamic interaction information, thereby improving the ability to recognize detailed traffic scenes.

[0177] Step 220: Input the perceived information and the two-dimensional label of the scene category into the scene recognition model to be trained, so that the scene recognition model to be trained learns the scene category label of the second training data, and obtains the trained scene recognition model, which is used as the scene recognition model.

[0178] Figure 6 As shown Figure 5 The diagram shows the training process of the scene recognition model in the vehicle decision-making and planning method.

[0179] like Figure 6 As shown, the scene recognition model is trained using the second training data with two-dimensional labels, and the scene recognition result, such as the predicted scene label, is output. Based on the two-dimensional label of the scene category and the scene recognition result, the scene classification loss is determined as follows: the loss function is used to calculate the loss value between the predicted scene label and the two-dimensional label of the real scene category; the convergence state of the loss value between the current iteration round and the previous round determines whether to continue iterative training; when the loss value converges to a stable state and no longer decreases, the iteration is stopped and the training is completed.

[0180] In the first example, the scene recognition model to be trained uses a multi-layered neural network structure to perform deep mining and feature learning on the data, continuously adjusting the parameters of the scene recognition model to minimize the error between the predicted scene category and the labeled two-dimensional label, and finally converges to obtain a scene recognition model that can accurately identify the scene category.

[0181] In the second example, during the training phase of the scene recognition model to be trained, the cross-entropy loss function is used to continuously adjust the model parameters through the backpropagation algorithm, so that the scene category probability distribution output by the scene recognition model to be trained is as close as possible to the labeled two-dimensional labels, until the scene recognition model to be trained converges, thereby obtaining a scene recognition model that can accurately identify the basic scene and its dynamic interaction information.

[0182] The scene recognition model described in this paper may include, but is not limited to, at least one of the following: Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Transformer model and its variants, hybrid models of CNN and LSTM, and fusion architectures of Transformer model and CNN. Any scene recognition model that takes perceptual information or its fused vector information as input, can be trained offline to obtain a scene recognition model, and can output scene category information online and collaborate with a decision-making and planning model to solve the problems of decision-making and planning models being unable to adaptively match scene decision preferences and the disconnect between perceptual and scene information, falls within the scope of protection of this invention.

[0183] In this embodiment, the second training data reflects the detailed traffic scene in which the vehicle is located, and the two-dimensional labeling method enables the scene recognition model to be trained to simultaneously learn the static attributes and dynamic interaction features of the scene, thereby more accurately understanding and recognizing complex and ever-changing traffic scenes. Then, after inputting the perceptual information and two-dimensional labels into the scene recognition model to be trained, a scene recognition model capable of accurately identifying scene categories is obtained. Thus, the training process of the scene recognition model utilizes a large amount of second training data under different scenarios and dynamic interaction conditions to improve the generalization ability and recognition accuracy of the trained scene recognition model in practical applications.

[0184] Furthermore, the aforementioned scene recognition model includes a feature extraction backbone network and a scene classification head.

[0185] Step 310: Extract environmental features from the perceived information through the feature extraction backbone network.

[0186] The aforementioned feature extraction backbone network can employ mature deep learning network architectures such as ResNet (Residual Network) and MobileNet (Lightweight Convolutional Neural Network). These architectures possess powerful feature abstraction capabilities, enabling them to extract environmental features from complex perceptual information (such as images and point clouds) layer by layer, from lower to higher levels, as well as the spatial relationships and contextual information between these elements. Thus, the feature extraction backbone network is responsible for extracting environmental features from sensor data, while the scene classification head can output the scene category probability distribution.

[0187] Step 320: Using the scene classification head, output the scene category probability distribution based on the environmental features.

[0188] The aforementioned scene classification head consists of fully connected layers or convolutional layers, etc. It receives environmental features from the output of the feature extraction backbone network and maps them into a scene category probability distribution through activation functions such as softmax. Each element in this probability distribution corresponds to the predicted probability of a specific scene category, thereby realizing the probabilistic judgment of the scene category in which the current vehicle is located.

[0189] Furthermore, by comparing the scene category probability distribution output by the scene recognition model with a preset scene judgment threshold, the specific scene category in which the vehicle is currently located can be accurately determined. This not only effectively addresses complex and ever-changing traffic environments but also improves the accuracy and robustness of recognition through continuous optimization of model parameters, ensuring that the vehicle responds appropriately in different scenarios.

[0190] In related technologies, when the model input parameters do not include scene category labels, the model relies solely on perceptual information to autonomously determine scene attributes, making it difficult to efficiently distinguish the semantic differences of the same perceptual information in different scenarios. For example, when the perceptual information indicates that a vehicle in the adjacent lane has activated its turn signal, the model cannot quickly determine whether the behavior is a normal lane change on a highway (where the vehicle can choose to accelerate / not decelerate to prohibit the lane change, or decelerate to allow the other vehicle to change lanes) or a lane merge or a lane change based on navigation information (which requires deceleration in advance and leaving room for avoidance), which can easily lead to confusion in decision-making logic and result in insufficient understanding of complex scenarios after training.

[0191] Compared to related technologies that struggle to efficiently distinguish the semantic differences of the same perceived information in different scenarios, leading to confusion in decision-making logic and insufficient understanding of complex scenarios by the trained model, in this embodiment, by independently training the scene recognition model, the mapping relationship between the scene and multi-sensor features can be learned in advance, generating standardized and high-precision scene category labels.

[0192] Thus, an independent scene recognition model is trained offline, employing a supervised learning and classification loss-guided strategy. Based on real-vehicle multi-sensor data and two-dimensional labels, it learns the mapping relationship between scenes and features, outputting standardized scene category labels. This scene recognition model can pre-model scene features, avoiding the inefficiency and bias of models autonomously mining scene information from perceived information. It can distinguish semantic differences of the same perceived information in different scenes, avoiding confusion in decision logic, and enabling the trained scene recognition model to improve its understanding of complex scenes. Furthermore, it facilitates the training and inference of subsequent decision planning models and ensures that these models have unified and reliable scene categories.

[0193] In combination with the above Figure 6 As shown, step 310 above may further include step 311, and step 320 above may further include steps 321 to 323: Step 311: Extract the static element tensor and dynamic element tensor of the current scene from the perceived information through the feature extraction backbone network.

[0194] The feature extraction backbone network can employ deep convolutional neural network architectures such as, but not limited to, ResNet and EfficientNet. These architectures use multiple layers of convolution, pooling, and non-linear activation operations to progressively abstract features at different levels from the raw perceptual information. In this way, the feature extraction backbone network parses static and dynamic element tensors from complex perceptual information.

[0195] Static elements are used to determine the spatial location and morphological stability of static elements, providing basic information for scene recognition. The static element tensor includes relatively fixed objects in the scene, such as road markings, traffic lights, buildings, and trees. In this way, the features of these elements have strong spatial location and morphological stability, constructing detailed road segment information of the basic scene.

[0196] The aforementioned dynamic element tensor is used to capture the multi-dimensional dynamic attributes of moving objects, such as the speed of vehicles and the direction of movement of pedestrians, reflecting the real-time changes in the current scene.

[0197] Specifically, this is achieved using the static element feature tensor of the perceived data, namely: and the dynamic element feature tensor, that is: Given the input, output the probability distribution of scene categories.

[0198] An architecture combining dual-branch encoding and Transformer fusion with a classification head is adopted, and cross-entropy loss is used for training, i.e.: .

[0199] in, One-hot encoding for the two-dimensional labels of scene categories, The scene probability output by the scene recognition model. This represents the total number of scene categories. After training, the scene recognition model can output scene numerical labels.

[0200] Step 321: Encode the static element tensor and the dynamic element tensor respectively to obtain static features and dynamic features.

[0201] In the encoding process of step 321, spatial attention mechanism can be used to encode static element tensors to highlight the feature weights of key static areas, such as stop lines and signs at road intersections; for dynamic element tensors, time series encoding methods, such as LSTM (Long Short-Term Memory) or Transformer encoder structures, can be combined to capture the motion trends and temporal dependencies of dynamic objects.

[0202] Step 322: The static features and the dynamic features are fused to obtain fused features.

[0203] The fusion process in step 322 can be carried out by means of feature splicing, weighted summation, attention fusion or gating mechanism to fuse static features with dynamic features.

[0204] For example, through an adaptive gating unit, the contribution ratio of static and dynamic features in the fused features can be dynamically adjusted according to the degree of dynamic change in the current scene. When there are many dynamic objects in the scene and their movements are complex, the weight of dynamic features is increased, and vice versa.

[0205] Step 323: Map the fused features to probability distributions of multiple specific scene categories.

[0206] In step 323 above, the fused features are mapped to probability distributions for various specific scene categories. This can be achieved using a fully connected layer in conjunction with the Softmax activation function. Each probability value in the probability distribution corresponds to the confidence level of the fused feature belonging to a specific scene category. For example, if, after mapping, the probability value of being overtaken in an urban road scene is 0.85, the probability value of being overtaken in a highway scene is 0.12, and the probability values ​​for other scenes are lower, it indicates that the current scene is most likely to be identified as an urban road scene where overtaking is possible.

[0207] In this embodiment, by encoding the different characteristics of static and dynamic elements separately, the feature representation of key static regions and the ability to capture the motion trends of dynamic objects are effectively enhanced. The subsequent fusion step ensures that the fused features comprehensively and selectively reflect the overall information of the current scene. Finally, the confidence level of the current scene belonging to different target categories is intuitively given, thereby realizing the mapping from raw perceived information to specific scene categories, facilitating subsequent vehicle control.

[0208] Step 321 above may include, but is not limited to, the following steps 3211 to 3213: Step 3211: Based on the attention mechanism, determine the association weights between the static features and the dynamic features.

[0209] The aforementioned attention mechanism captures the intrinsic relationship between static and dynamic features.

[0210] Specifically, in step 3211, the mutual influence between static and dynamic features in different dimensions and spatial locations is analyzed through the attention mechanism, and the correlation weight between the two is calculated.

[0211] These association weights can quantify the importance of static and dynamic features to the current scene recognition.

[0212] For example, in detailed road segment information, the association weight of dynamic features (such as at least one of vehicles approaching from the side or pedestrians crossing the road) will be increased accordingly to highlight their impact on the current scene category judgment.

[0213] In relatively simple scenarios such as highways, the association weight of static features (such as at least one of road type markings and guardrail styles) accounts for a large proportion.

[0214] Step 3212: Based on the association weight, the static features and the dynamic features are weighted by attention to obtain the weighted static features and the weighted dynamic features.

[0215] For features with high correlation weights, their contribution in the subsequent fusion process will be amplified, while the contribution of features with low weights will be weakened accordingly, thereby achieving the focus on key information and the suppression of noise information.

[0216] Step 3213: The weighted static features and the weighted dynamic features are fused to obtain the fused features.

[0217] The feature extraction network in this paper is used to extract basic scene features and dynamic interaction features from perceptual information. The aforementioned fusion network deeply fuses these two types of features, and the classification network is used to output a scene category probability distribution based on the fused features. Each element in this probability distribution corresponds to a probability of a two-dimensional scene category composed of basic scene and dynamic interaction information.

[0218] 1. Real-time collection of multi-source data from actual vehicles is as follows: The actual vehicle is equipped with sensors such as cameras, lidar, and millimeter-wave radar, which simultaneously collect perception information about the current driving environment; this perception information includes perception data and vehicle status data; among which, The aforementioned perception data may include, but is not limited to, raw data of static elements (such as lane line positions, roadside contours, traffic signs, etc.) and raw data of dynamic elements (such as pedestrian positions and speeds, adjacent vehicle movement status, non-motorized vehicle trajectories, etc.).

[0219] The aforementioned vehicle status data may include, but is not limited to, operating parameters such as vehicle speed, acceleration, steering angle, and gear position.

[0220] 2. The preprocessing and feature extraction of the sensory data are as follows: The collected raw data is standardized and preprocessed to generate feature tensors with the same format as those used in the offline training phase. The static element feature tensor is The preprocessing method is completely consistent with the static feature extraction logic during scene recognition model training; The feature tensor of dynamic elements is The same temporal sampling frequency and feature encoding rules as offline training are used.

[0221] 3. Real-time scene category recognition is as follows: The preprocessed perceptual feature tensor is input into the scene recognition model. The scene recognition model, through offline training and learning of scene classification logic, outputs the probability distribution of the current scene type. The category corresponding to the highest probability is selected as the final scene category, and a corresponding one-hot encoded vector is generated. .

[0222] 4. The fusion of scene categories and perceived information, and the decision-making reasoning of the PDP model are as follows: Fusion of scene category and perception information: real-time scene category vector With perceptual feature tensor , Perform dimensional alignment and feature concatenation to generate fused features. The fusion method is consistent with the input fusion logic of the weight parsing network in the offline training stage.

[0223] Adaptive decision output: fused features Input the PDP model, which learns basic scene categories, perceptual detail information, and loss term weight association logic based on the offline training phase, and directly outputs decision planning instructions adapted to the current environment.

[0224] 5. The safety fallback mechanism is triggered as follows: To improve the reliability of the inference stage, a scene recognition confidence threshold judgment is added: if the maximum probability value output by the scene recognition model is lower than the preset threshold, the current scene is determined to be an unknown scene, and the PDP model automatically switches to a conservative decision-making mode to prioritize improving safety redundancy, such as reducing vehicle speed and increasing following distance, to ensure the safety of actual vehicle driving.

[0225] In this embodiment, deep integration is performed based on the weighted information of both, enabling the fused feature vector to simultaneously retain the structural information of the static environment and the behavioral trend information of dynamic objects, resulting in more comprehensive and discriminative scene classification. This allows for adaptive adjustment of feature contributions based on the actual association between static and dynamic elements in the scene, further improving the quality of the fused features and the accuracy of scene recognition.

[0226] Combination Figure 5 As shown, step 110 above may include, but is not limited to, step 111, and step 110 above may include, but is not limited to, step 121: Step 111: Obtain current perception information of the current scene collected by multiple sensors of the actual vehicle; the current perception information includes the current scene and the current dynamic interaction information of the current scene.

[0227] Step 121: Synchronously input the current scene and the current dynamic interaction information from the multiple sensors into the scene recognition model to output the scene category probability distribution.

[0228] In related technologies, scene categories are not used as model input during the real-vehicle inference stage, preventing the model from accessing the scene and decision-making logic learned during offline training. For example, when a real vehicle approaches an "unprotected left-turn intersection," the model cannot quickly switch to a "safety-first" decision mode based on the scene category, and instead continues to use the decision-making logic of conventional roads, which can easily lead to aggressive driving and affect the adaptability and safety of decisions in real-world scenarios.

[0229] Compared to related technologies that do not use scene categories as model input, which can easily lead to aggressive driving problems and affect the decision adaptability and safety in real-world scenarios, this application embodiment achieves end-to-end real-time response for scene perception and decision adaptation, enabling the decision planning model to call the offline trained scenario-based decision logic, while avoiding the risk of scene recognition failure through safety fallback decisions, thus comprehensively improving the decision adaptability and safety of real-world vehicle operation.

[0230] In this embodiment, during actual vehicle operation, multi-sensor data is synchronously input into the scene recognition model, which outputs a current scene category flag, thereby improving the accuracy of scene category recognition. Subsequently, the current scene category flag is further fed into the decision-making and planning model in real time, triggering the scene adaptation decision learned during the offline training phase of the model, achieving end-to-end response for scene perception decision adaptation. Simultaneously, a confidence threshold judgment of the scene recognition result is added; when the confidence level falls below the threshold, the system automatically switches to a safety fallback decision to ensure driving safety.

[0231] Furthermore, the offline-trained scene recognition model and decision-making planning model are deployed collaboratively to construct an integrated online inference process. By embedding the scene recognition model into the online inference architecture, real-time acquisition and input of scene categories are achieved, allowing the decision-making planning model to directly call the scene and decision-making association logic learned offline. Real-time multi-sensor data from the vehicle is input into the scene recognition model, and the output scene labels synchronously trigger the scene adaptation decision of the decision-making planning model.

[0232] As an embodiment of this application, the method further includes: constructing two-dimensional labels for scene categories using steps 410 to 430 as follows: Step 410: Obtain perception information for each scene category. This perception information is obtained by collecting data from the vehicle's sensors in the current scene.

[0233] Step 420: Based on the correlation between each basic scene and the corresponding dynamic interaction information within the basic scene, the perceived information of each scene category is annotated in two dimensions: the basic scene dimension and the dynamic dimension, to obtain annotated data. Thus, based on the two-dimensional classification of basic scenes and dynamic interactions, the collected perceived information is type-labeled. First, the basic scene dimension is identified, then the dynamic interaction dimension is identified, ultimately forming annotated data for basic scenes and dynamic interactions. Finally, the annotated data is digitally labeled.

[0234] Step 430: Digitize the labeled data to obtain two-dimensional labels for the scene categories corresponding to the dynamic interactive information in the basic scene.

[0235] In this embodiment, by constructing two-dimensional labels for scene categories, more detailed scene representations can be achieved. Furthermore, the two-dimensional labels provide richer and more discriminative sample data for the subsequent training of the scene recognition model, thereby improving the model's ability to recognize complex dynamic scenes. This breaks through the limitations of traditional single-dimensional scene segmentation, resulting in more comprehensive and detailed scene representation.

[0236] The vehicles mentioned in this article may include, but are not limited to, one or more of heavy trucks, light commercial vehicles, and passenger cars.

[0237] The electronic devices mentioned in this article may include, but are not limited to, in-vehicle terminals connected to a vehicle. In-vehicle terminals connected to a vehicle may be, but are not limited to, body processors, center consoles, or automotive HUDs (Head and Up Displays).

[0238] Figure 7 The diagram shown is a structural schematic of the electronic device 50 provided in an embodiment of this application.

[0239] like Figure 7 As shown, the electronic device 50 includes one or more processors 51 for implementing the training method of the decision planning model as described above or the vehicle decision planning method as described above.

[0240] In some embodiments, electronic device 50 may include storage medium 59. For example, computer-readable storage medium may store a program that can be invoked by processor 51, and may include non-volatile storage medium. In some embodiments, electronic device 50 may include memory 58 and interface 57. In some embodiments, electronic device 50 may also include other hardware depending on the specific application.

[0241] The computer-readable storage medium of this application embodiment stores a program that, when executed by the processor 51, is used to implement the training method of the decision planning model as described above or the vehicle decision planning method as described above.

[0242] This application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the method described in any of the preceding claims.

[0243] This application also provides a computer program stored in a computer-readable storage medium, such as... Figure 7 The storage medium 59, and when the processor executes the computer program, causes the processor 51 to perform the method described above.

[0244] This application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented using any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0245] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

[0246] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element qualified by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A training method for a decision-making programming model, characterized in that, include: Based on the two-dimensional labels of scene categories in the first training data, and the perception information collected by at least one sensor of the actual vehicle, the input data for the decision planning model to be trained and the weight parsing network are generated. The two-dimensional labels for the scene categories are used to mark each basic scene and its corresponding dynamic interactive information corresponding to the scene category; The input data is respectively input into the decision planning model to be trained and the weight parsing network, so that the decision planning model outputs the decision result under the scene category, and the weight parsing network learns the two-dimensional label of the scene category and the perception information, and outputs the loss term weights that are dynamically matched with the decision result; Based on the loss term and the weights of the matched loss term, a well-trained decision planning model is obtained.

2. The training method for the decision-making programming model as described in claim 1, characterized in that, The weight parsing network learns the two-dimensional labels of the scene category and the perceptual information, and outputs loss term weights that are dynamically matched with the decision result, including: The weight parsing network deeply integrates the two-dimensional labels of the scene category with the perceived information to generate differentiated weights that adapt to different environmental details of the same scene category, which serve as loss term weights that are dynamically matched with the decision results.

3. The training method for the decision-making programming model as described in claim 2, characterized in that, The weight parsing network includes a graph neural network; the perceptual information includes static elements and dynamic elements. The step of deeply fusing the two-dimensional labels of the scene category with the perceived information to generate differentiated weights that adapt to different environmental details of the same scene category, and using these weights as loss term weights that are dynamically matched with the decision result, includes: The two-dimensional labels, static elements, and dynamic elements of the scene categories in the perceived information are used to construct an association graph. The graph nodes of the association graph include scene label nodes for representing scene category attributes, as well as static element nodes and dynamic element nodes. The edge weights of the association graph are constructed based on the matching degree between the graph nodes and the scene, as well as the spatial conflict relationship and / or motion conflict relationship between the graph nodes. By dynamically updating the edge weights, we can obtain the perceived changes in different environmental details. Global features that combine scene category attributes and environmental details from the same scene are used to generate differentiated weights.

4. The training method for the decision-making programming model as described in claim 3, characterized in that, The global features that integrate scene category attributes and environmental details of the same scene are used to generate differentiated weights, including: By aggregating the dynamic feature feedback and preference guidance of all nodes through neighborhood features, the system outputs global features that integrate scene category attributes and environmental details of the same scene, generating differentiated weights.

5. The training method for the decision-making programming model as described in any one of claims 1 to 4, characterized in that, The number of loss item weights dynamically matched with the decision result is determined in real time based on the scenario category; When there are multiple loss item weights that are dynamically matched with the decision result, the loss item includes multiple loss items accordingly; The weight parsing network learns the two-dimensional labels of the scene category and the perceptual information, and outputs loss term weights that are dynamically matched with the decision result, including: The weight parsing network learns the two-dimensional labels of the scene category and the perception information, and outputs the weights of each loss term that are dynamically matched with the decision result; The trained decision-making and programming model includes: Based on each loss term and its weight dynamically matched with the decision result, the loss value of each loss term between the decision result and the actual result is determined to obtain the total loss value. When the total loss value converges to a stable value, the iteration stops, and the trained decision programming model is obtained.

6. The training method for the decision-making programming model as described in any one of claims 1 to 4, characterized in that, The number of loss items is determined in real time based on the scenario category; When the number of loss term weights dynamically matched with the decision result is 1, the corresponding loss term includes one loss term; The trained decision planning model, based on the loss term and the matched loss term weights, includes: Based on a loss term and a loss term weight that is dynamically matched with the decision result, the loss value of the loss term between the decision result and the actual result is determined respectively. When the loss value converges to a stable state, the iteration stops, and the trained decision programming model is obtained.

7. The training method for the decision-making programming model as described in any one of claims 1 to 4, characterized in that, The input data for generating the decision planning model and weight parsing network to be trained, based on the two-dimensional labels of scene categories in the first training data and the perception information collected by at least one sensor of the actual vehicle in the first training data, includes: Based on the two-dimensional labels of scene categories in the first training data and the encoded features of the perceived information, a fusion feature containing environmental information and scene context is obtained. The fused features are then input into the decision planning model and the weight parsing network to be trained, respectively.

8. A vehicle decision-making and planning method, characterized in that, include: Acquire current perception information in the current scene collected by the vehicle's sensors; Based on the current perceived information, the current scene category is identified; The current scene category and the current perception information are input into the trained decision planning model so that the trained decision planning model outputs planning control instructions that are adapted to the current scene category according to the decision preference of the current scene category. The trained decision-making and programming model is a decision-making and programming model trained using the training method for decision-making and programming models as described in any one of claims 1 to 7.

9. The vehicle decision-making and planning method as described in claim 8, characterized in that, The step of identifying the current scene category based on the current perceived information includes: The current perceived information is input into the scene recognition model to output the scene category probability distribution; the scene recognition model is trained based on training data and two-dimensional labels of each basic scene and its corresponding dynamic interaction information corresponding to the scene category. Based on the probability distribution of the scene categories, the current scene category containing the basic scene and its corresponding dynamic interaction information is determined.

10. The vehicle decision-making and planning method as described in claim 9, characterized in that, The step of determining the current scene category, which includes the basic scene and its corresponding dynamic interaction information, based on the scene category probability distribution includes: The scene category with the highest probability in the scene category probability distribution is taken as the current scene category.

11. The vehicle decision-making and planning method as described in claim 9, characterized in that, The step of taking the scene category with the highest probability in the scene category probability distribution as the current scene category includes: when there are multiple scene categories with the highest probability in the scene category probability distribution, taking any scene category with the highest probability as the current scene category; or, The step of taking the scene category with the highest probability in the scene category probability distribution as the current scene category includes: when there are multiple scene categories with the highest probability in the scene category probability distribution, taking the scene category with the highest security weight among the multiple scene categories with the highest probability as the current scene category.

12. The vehicle decision-making and planning method as described in claim 8, characterized in that, The step of determining the current scene category containing the basic scene and its corresponding dynamic interaction information based on the scene category probability distribution includes: taking the scene category corresponding to the scene category probability greater than the confidence threshold as the current scene category; or, Based on the scene category probability distribution, the current scene category containing the basic scene and its corresponding dynamic interaction information is determined, including: If the scene category is determined to be a scene category that is less than the confidence threshold, then the scene category that is less than the confidence threshold is taken as the current scene category.

13. The vehicle decision-making and planning method as described in any one of claims 8 to 12, characterized in that, The method further includes: the scene recognition model is trained in the following manner: Acquire second training data; the second training data includes: perception information collected by at least one sensor of the actual vehicle, and used to label each basic scene with a two-dimensional label corresponding to the scene category of the dynamic interaction information; The perceived information and the two-dimensional label of the scene category are input into the scene recognition model to be trained, so that the scene recognition model to be trained learns the scene category label of the second training data, and a trained scene recognition model is obtained, which is used as the scene recognition model.

14. The vehicle decision-making and planning method according to any one of claims 8 to 12, characterized in that, The scene recognition model includes a feature extraction backbone network and a scene classification head; Environmental features are extracted from the perceived information through the feature extraction backbone network. Based on the environmental features, the scene classification head outputs the probability distribution of the scene category.

15. The vehicle decision-making and planning method as described in claim 14, characterized in that, The step of extracting environmental features from the perceived information through the feature extraction backbone network includes: Through the feature extraction backbone network, the static element tensor and dynamic element tensor of the current scene are extracted from the perceived information; The step of outputting a scene category probability distribution based on the environmental features using the scene classification head includes: The static element tensor and the dynamic element tensor are encoded respectively to obtain static features and dynamic features; The static features and the dynamic features are fused to obtain the fused features; The fused features are mapped to a probability distribution of scene categories.

16. The vehicle decision-making and planning method as described in claim 15, characterized in that, The step of encoding the static element tensor and the dynamic element tensor respectively to obtain static features and dynamic features includes: Based on the attention mechanism, the association weights between the static features and the dynamic features are determined; Based on the association weights, attention weights are applied to the static features and the dynamic features to obtain weighted static features and weighted dynamic features. The weighted static features and the weighted dynamic features are fused to obtain the fused features.

17. The vehicle decision-making and planning method as described in any one of claims 8 to 12, characterized in that, The acquisition of current perception information in the current scene collected by the vehicle's sensors includes: Acquire current perception information in the current scene from multiple sensors of the actual vehicle; the current perception information includes the current scene and the current dynamic interaction information in the current scene. The step of inputting the current perceived information into the scene recognition model to output a scene category probability distribution includes: The current scene and the current dynamic interaction information from the multiple sensors are synchronously input into the scene recognition model to output the scene category probability distribution.

18. An electronic device, characterized in that, It includes one or more processors for implementing the training method for the decision planning model as described in any one of claims 1 to 7 or the vehicle decision planning method as described in any one of claims 8 to 17.

19. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the training method for the decision planning model as described in any one of claims 1 to 7 or the vehicle decision planning method as described in any one of claims 8 to 17.

Images