Vehicle and travel trajectory generation method and control method

Abstract

The application discloses a kind of vehicle and driving track generation method and control method related to intelligent driving technical field, the method includes: obtaining the visual data of vehicle surrounding environment, and environment object identification is carried out to visual data;Dynamic semantic behavior graph is constructed based on the environment object identified;Language instruction is obtained outside input, and language instruction is carried out semantic analysis, and global semantic feature is extracted;Global semantic feature is used to update dynamic semantic behavior graph;Based on global semantic feature and the updated dynamic semantic behavior graph, the space-time characteristic reasoning of interactive behavior relationship is carried out, and the behavior strategy of vehicle is obtained;Based on behavior strategy and the risk distribution information of environment object, target driving track satisfying semantic constraint condition is generated.The method can realize the track generation and control of vehicle based on semantic constraint under complex dynamic environment, improve the safety and environmental adaptability of vehicle driving.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a method for generating and controlling a vehicle and its driving trajectory. Background Technology

[0002] With the continuous development of driving technology, low-speed autonomous vehicles are being increasingly widely used in scenarios such as park logistics, campus delivery, and inspection. In related technologies, the behavioral decisions of low-speed autonomous vehicles typically rely on pre-defined rule systems, finite state machine models, or traditional path planning algorithms. While these methods are structurally relatively clear, they often require continuous addition and adjustment of rules for different scenarios when facing complex and ever-changing real-world environments. This leads to a rapid increase in system complexity, high maintenance costs, and difficulty in adapting to the uncertainties brought about by environmental changes.

[0003] On the other hand, related visual perception technologies typically focus on the detection and localization of target objects in the environment. The output information mainly includes low-level features such as target category, location, and motion state, lacking the ability to express the relationships between objects in the environment. For example, interactive information such as the avoidance relationship between vehicles and pedestrians, and the following relationship between vehicles, is difficult to reflect directly through existing perception results, thus limiting the depth of behavioral decision-making's understanding of environmental semantics.

[0004] Furthermore, in trajectory planning, most methods rely primarily on geometric path feasibility and dynamic constraints, focusing on fundamental issues such as obstacle avoidance and accessibility, but lack the ability to effectively express semantic constraints within the environment. For example, when facing complex scenarios such as pedestrians and intersections, they struggle to reflect the differentiated processing needs of different objects or scenario conditions, thus affecting the rationality and adaptability of vehicle behavior to some extent. Therefore, the related technologies still have shortcomings in terms of behavioral expression and decision-making flexibility in complex environments. Summary of the Invention

[0005] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, the purpose of this invention is to propose a vehicle and driving trajectory generation and control method to improve the semantic expression capability of autonomous vehicle behavior decision-making and the rationality of trajectory generation in complex environments.

[0006] To achieve the above objectives, a first aspect of the present invention provides a method for generating a vehicle driving trajectory, comprising: Acquire visual data of the vehicle's surrounding environment and perform environmental object recognition on the visual data; A dynamic semantic behavior graph is constructed based on the identified environmental objects; the dynamic semantic behavior graph includes nodes and edges, the nodes correspond one-to-one with the environmental objects, and the edges are used to represent the interactive behavior relationships between the vehicle and each environmental object. Obtain externally input language instructions, perform semantic parsing on the language instructions, and extract global semantic features; The dynamic semantic behavior graph is updated using the global semantic features; Based on the global semantic features and the updated dynamic semantic behavior graph, spatiotemporal feature reasoning is performed on the interaction behavior relationship to obtain the vehicle's behavior strategy. Based on the behavioral strategy and the risk distribution information of the environmental objects, a target driving trajectory that meets the semantic constraints is generated.

[0007] In addition, the method of the above embodiments of the present invention may also have the following additional technical features: According to an embodiment of the present invention, constructing a dynamic semantic behavior graph based on the identified environmental objects includes: Each of the environmental objects is mapped to a unique node in the dynamic semantic behavior graph, and the spatial state information and semantic feature vector of each environmental object are used as attributes of the corresponding node. Based on the relative motion state and semantic feature similarity between the vehicle and each node, directed edges between the vehicle and each node are established by combining rule matching and similarity calculation, and each edge is assigned an initial weight and behavioral relationship; the behavioral relationship includes avoidance, following, yielding, and stopping.

[0008] According to an embodiment of the present invention, updating the dynamic semantic behavior graph using the global semantic features includes: A cross-attention mechanism is employed, using the global semantic features as the query and the spatial semantic features of each node in the dynamic semantic behavior graph as the key and value, to calculate the attention weight; the spatial semantic features include spatial state information and semantic feature vectors. The spatial semantic features of each node are updated by weighting according to the attention weights to obtain the updated spatial semantic features, thus completing the update of the dynamic semantic behavior graph.

[0009] According to an embodiment of the present invention, the spatiotemporal feature reasoning of the interaction behavior relationship includes: The updated dynamic semantic behavior graph is transformed into structured graph data, and the global semantic features are mapped into semantic guidance vectors representing instruction guidance. The structured graph data and the semantic guidance vector are initially fused to obtain the initial fusion features of the nodes; Based on the initial weights of the edges and the similarity of the spatial semantic features, and by introducing the semantic guidance vector to constrain the update direction of the spatial semantic features, the attention weights between each node are calculated using a multi-head attention mechanism. The attention weights are used to aggregate the spatial semantic features of adjacent nodes in a weighted manner, and combined with the initial fusion features of the current node, to realize the transmission of interactive information between nodes.

[0010] According to an embodiment of the present invention, the spatiotemporal feature inference of the interaction behavior relationship further includes: The spatial semantic features of the previous inference cycle are jointly processed with the current weighted aggregated spatial semantic features, and gating is performed based on the semantic guidance vector to retain feature components consistent with the language instructions and suppress irrelevant feature components. The spatial semantic features of the current inference cycle are updated based on the filtered feature results.

[0011] According to an embodiment of the present invention, the risk distribution information of the environmental object is obtained in the following manner: Assign preset risk weights based on the category of the environmental object; The risk score is calculated using an exponential decay function based on the distance between the vehicle and the environmental object and the risk weight. By summarizing the risk scores of each environmental object, a semantic risk field representing the distribution of environmental hazard levels is constructed.

[0012] According to one embodiment of the present invention, generating a target driving trajectory that satisfies semantic constraints includes: Based on the semantic risk field, trajectory space sampling is performed to generate a set of candidate driving trajectories; Select candidate trajectories from the candidate driving trajectory set that avoid areas with risk scores higher than a preset threshold as the initial driving trajectory; An objective function is constructed that includes smoothness loss, risk field penalty loss, and behavioral policy loss. The objective function is then solved using a constrained optimization algorithm to obtain the target driving trajectory.

[0013] According to an embodiment of the present invention, solving the objective function based on the constrained optimization algorithm includes: Using the trajectory parameters of the initial driving trajectory, the risk score output by the semantic risk field, the position change information of the environmental object, and the constraint deviation of the behavior strategy as the state space, and the adjustment amount of the trajectory parameters as the action space, the initial driving trajectory is adjusted by iteratively optimizing the objective function to generate the target driving trajectory; the adjustment amount includes at least one change in trajectory point position, speed, and steering angle.

[0014] To achieve the above objectives, a second aspect of the present invention provides a vehicle control method, comprising: The target driving trajectory of the vehicle is generated using a vehicle driving trajectory generation method. Control the vehicle to travel along the target trajectory.

[0015] To achieve the above objectives, a third aspect of the present invention provides a vehicle including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0016] The vehicle and driving trajectory generation and control methods of this invention, through visual perception of the vehicle's surrounding environment and identification of environmental objects, construct a dynamic semantic behavior graph containing objects and their interaction relationships, thereby achieving a structured expression of multi-subject interaction relationships in complex traffic environments. Based on this, global semantic features corresponding to language commands are introduced to semantically update the dynamic semantic behavior graph, enabling the vehicle to integrate external semantic constraint information on top of environmental perception information. Subsequently, by performing spatiotemporal feature reasoning on the updated dynamic semantic behavior graph, behavioral strategies that meet task requirements can be generated while considering the dynamic interaction relationships between objects and incorporating semantic guidance. By incorporating the risk distribution information of environmental objects and introducing semantic constraints during trajectory generation, the generated driving trajectory not only meets geometric feasibility and safety requirements but also reflects a comprehensive response to environmental risks and semantic commands, thereby improving the vehicle's behavioral decision-making ability and trajectory planning rationality in complex scenarios. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a method for generating vehicle driving trajectory in one embodiment; Figure 2 This is a schematic diagram illustrating the construction process of a dynamic semantic behavior graph in one embodiment; Figure 3 This is a schematic diagram of the update process of a dynamic semantic behavior graph in one embodiment; Figure 4 This is a flowchart illustrating spatiotemporal feature reasoning in one embodiment; Figure 5 This is a schematic diagram of the target driving trajectory generation process in one embodiment; Figure 6 This is a schematic diagram of vehicle data processing in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] The implementation details of the technical solutions of the embodiments of the present invention are described in detail below.

[0020] In one embodiment, such as Figure 1 The diagram illustrates a method for generating vehicle driving trajectories, which may include the following steps: Step S101: Obtain visual data of the vehicle's surrounding environment and perform environmental object recognition on the visual data.

[0021] Visual data refers to perceptual data used to characterize the state of the vehicle's surrounding environment. It can be acquired through image acquisition devices placed around the vehicle, such as image or video data captured by in-vehicle cameras. After acquiring the visual data, it undergoes preprocessing operations, including distortion correction and viewpoint alignment of multi-view data, so that data from different viewpoints can be expressed in a unified coordinate system, thereby obtaining standardized visual data.

[0022] Based on this, visual data is analyzed and processed to identify various objects in the environment. During the identification process, target regions in the visual data are detected and features are extracted to determine the categories of environmental objects and their corresponding spatial state information. Environmental objects include pedestrians, vehicles, obstacles, road boundaries, and traffic signs; spatial state information includes location coordinates, movement speed, and direction of movement. Simultaneously, features are extracted for each environmental object to obtain semantic feature vectors representing its semantic attributes. The spatial state information is then fused with the semantic feature vectors to form a semantic information representation of the environmental object.

[0023] It should be noted that the vehicles are low-speed autonomous vehicles operating within a limited area, with a speed lower than a preset threshold, such as unmanned vehicles used in park delivery, inspection, or short-distance transportation scenarios.

[0024] Step S102: Construct a dynamic semantic behavior graph based on the identified environmental objects.

[0025] After identifying the environmental objects, each environmental object is organized as a basic unit of semantic expression, and a dynamic semantic behavior graph is constructed based on the interaction relationship between the environmental objects, so that the dynamic semantic behavior graph can provide a structured representation of the objects in the environment and their interaction relationship.

[0026] Dynamic semantic behavior graphs are a graph structure representation used to describe environmental objects and their interactive relationships. By using a graph structure to uniformly model discrete environmental object information and the interaction relationships between objects, a holistic representation of environmental semantic information can be achieved. In practical applications, dynamic semantic behavior graphs can be represented as graph structures that change over time. , where the set of nodes Each node in the set corresponds one-to-one with an environment object, representing the spatial state and semantic attributes of the corresponding environment object. (Edge set) Each edge in the diagram represents the interactive relationship between the vehicle and the corresponding environmental object, reflecting the way the vehicle interacts with the environmental object.

[0027] Since the spatial state of environmental objects (such as position, speed, and direction of movement) changes over time, the dynamic semantic behavior graph is dynamically updated during vehicle operation, so that the graph structure can continuously reflect the object state and interaction relationship in the environment at the current moment, thus providing a foundation for subsequent spatiotemporal feature reasoning.

[0028] In one embodiment, Figure 2 The diagram illustrates the construction process of a dynamic semantic behavior graph, which may include the following steps: Step S201: Map each environment object to a unique node in the dynamic semantic behavior graph, and use the spatial state information and semantic feature vector of each environment object as the attributes of the corresponding node.

[0029] After identifying the environmental objects, each environmental object is extracted from the set of environmental objects, and each environmental object is mapped to a node in the dynamic semantic behavior graph. The spatial state information and semantic feature vector corresponding to each environmental object are embedded as node attributes into the node structure to achieve a quantitative expression of the semantic attributes of the environmental object. By embedding spatial state information and semantic feature vectors together into the node structure, the node can not only identify the corresponding environmental object, but also reflect the motion characteristics and semantic attributes of that environmental object.

[0030] In practical applications, each node can be assigned a unique identifier to ensure distinguishability between nodes. Furthermore, nodes can be categorized and labeled based on the type of environmental objects to differentiate between different types of objects such as pedestrians, vehicles, and obstacles, thus providing a basis for matching subsequent interaction relationships and initializing weights.

[0031] Step S202: Based on the relative motion state and semantic feature similarity between the vehicle and each node, directed edges between the vehicle and each node are established by combining rule matching and similarity calculation, and each edge is assigned an initial weight and behavioral relationship.

[0032] Using vehicles as the initiating entity for interaction, the interaction relationship between vehicles and various environmental objects is analyzed by combining their relative positions, distances, speeds, and directions of motion. The type of interaction behavior is determined by combining rule matching and similarity calculation.

[0033] Rule matching is used to initially determine interaction relationships based on preset behavioral rules. Specifically, rule matching determines candidate behavioral relationship types based on the relative motion state between the vehicle and environmental objects, as well as the category information of the environmental objects. For example, when the environmental object is in front of the vehicle's path and is a pedestrian, the matching is to avoid or yield; when the environmental object is a vehicle traveling in the same direction and at a similar speed, the matching is to follow; when the environmental object is a static obstacle, the matching is to detour or stop.

[0034] Based on this, the similarity between the vehicle and the environmental objects is quantitatively evaluated through similarity calculation. The candidate behavioral relationships are then screened or corrected based on the similarity results to obtain the final interactive behavioral relationship type.

[0035] Based on the determination results of the above interaction behavior relationship types, directed edges are established between the vehicle corresponding node and each environmental object node, forming an edge set, specifically represented as follows:

[0036] in, This indicates the node corresponding to the vehicle. Represents the environment object node. This represents the behavioral relationship between a vehicle and environmental objects, and this behavioral relationship belongs to a predefined set of behavioral relationships. One of them includes yielding, following, giving way, and stopping.

[0037] At the same time, each directed edge is assigned an initial weight to characterize the strength of the interaction relationship, so as to reflect the degree of influence of the interaction relationship on vehicle behavior decisions.

[0038] Step S103: Obtain the language instructions from external input, perform semantic parsing on the language instructions, and extract global semantic features.

[0039] The system acquires external language commands, which describe the control requirements for vehicle behavior, including behavior type constraints, spatial constraints, and motion constraints. After acquiring the language commands, it performs semantic parsing to extract their corresponding semantic feature representations.

[0040] In the semantic parsing process, language instructions are transformed into word sequence representations, and these word sequences are encoded to obtain semantic vector representations for each word. During encoding, an attention mechanism is used to semantically model the word sequence representations to identify different semantic elements in the language instructions. Semantic elements may include behavior type information, spatial constraint information, and motion constraint information. Behavior type information characterizes the type of behavior the vehicle should perform (e.g., avoidance), spatial constraint information characterizes the vehicle's driving position or directional restrictions (e.g., keeping to the right), and motion constraint information characterizes the vehicle's speed or acceleration change requirements (e.g., deceleration).

[0041] Based on this, feature extraction and fusion are performed on semantic vectors, and behavior type information, spatial constraint information and motion constraint information are jointly encoded to generate a unified global semantic feature representation. This enables the global semantic features to simultaneously reflect the behavioral intentions and constraints in language instructions, thereby providing semantic guidance for subsequent updates of dynamic semantic behavior graphs and spatiotemporal feature reasoning.

[0042] Step S104: Update the dynamic semantic behavior graph using global semantic features.

[0043] After obtaining the global semantic features, the global semantic features are applied to the spatial semantic features of each node in the dynamic semantic behavior graph to update the semantic representation of each node. This allows the nodes to retain their original spatial state information while introducing semantic information related to language instructions, thereby achieving the fusion of visual perception information and language semantic information.

[0044] During the update process, the correlation between global semantic features and spatial semantic features of each node is modeled to determine the degree to which different nodes are affected by language instructions. Based on the correlation, the semantic representation of each node is adjusted differently so that nodes with higher semantic correlation with language instructions receive more significant feature updates, while nodes with lower correlation retain their original features or only receive weak updates.

[0045] Through the above update process, the spatial semantics in the dynamic semantic behavior graph not only reflects the spatial state and category attributes of the environmental objects, but also embodies the behavioral intentions and constraint information expressed by the language instructions, thus obtaining a semantically enhanced dynamic semantic behavior graph, which provides a foundation for subsequent semantically guided spatiotemporal feature reasoning.

[0046] In one embodiment, Figure 3 A schematic diagram of the update process for a dynamic semantic behavior graph may include the following steps: Step S301: Using a cross-attention mechanism, the global semantic features are used as the query, and the spatial semantic features of each node in the dynamic semantic behavior graph are used as the key and value to calculate the attention weight.

[0047] Based on the correlation between global semantic features and the spatial semantic features of each node in the dynamic semantic behavior graph, a cross-modal feature interaction process is constructed. The global semantic features are used as query vectors, and the spatial semantic features corresponding to each node are used as keys and values. The attention weight between each semantic feature and each node is calculated through a cross-attention mechanism.

[0048] By using attention weights, the degree of correlation between language semantics and different environmental objects is characterized, so that each environmental object can receive different levels of attention based on its relevance to the current language instruction, thereby providing a basis for subsequent spatial semantic feature updates.

[0049] Step S302: The spatial semantic features of each node are updated by weighting according to the attention weights to obtain the updated spatial semantic features, thus completing the update of the dynamic semantic behavior graph.

[0050] Based on attention weights, the spatial semantic features of each node are weighted and fused, so that spatial semantic features that are highly correlated with the current language instructions receive a larger update range, while spatial semantic features that are less correlated receive a smaller update range.

[0051] After updating each spatial semantic feature, the updated spatial semantic features are fused with the global semantic features to form a node-level joint representation that includes environmental object information and language semantic information, thereby achieving an overall update of the dynamic semantic behavior graph.

[0052] Step S105: Based on global semantic features and the updated dynamic semantic behavior graph, perform spatiotemporal feature reasoning on the interaction behavior relationship to obtain the vehicle's behavior strategy.

[0053] After obtaining the updated dynamic semantic behavior graph and global semantic features, spatiotemporal feature reasoning is performed on the interactive behavioral relationships between environmental objects. Spatiotemporal feature reasoning is used to comprehensively analyze the interactive relationships between the vehicle and various environmental objects based on the graph structure representation, combined with the spatial distribution relationships of environmental objects and their dynamic characteristics that change over time, thereby determining the vehicle's behavioral decision-making results in the current environment.

[0054] During the reasoning process, spatial semantic information, interaction relationships represented by edges, and global semantic features in the dynamic semantic behavior graph are jointly modeled. This ensures that the reasoning results not only reflect the interaction effects between environmental objects but also embody the behavioral intentions and constraints expressed by language instructions. By evaluating the degree of influence of interaction relationships between different environmental objects, the contribution of each interaction relationship to vehicle decision-making is determined, and a vehicle behavior strategy is generated based on the comprehensive evaluation results.

[0055] Among them, the behavior strategy is used to characterize the vehicle's behavior decision results under the current environmental state and semantic constraints, including the type of behavior that the vehicle should perform and its corresponding control trend information. The behavior type includes at least one of avoidance, following, yielding or stopping, and the control trend information includes at least one of speed adjustment trend, driving direction adjustment trend or path deviation trend.

[0056] Through the above processing, the output behavioral strategy can simultaneously integrate environmental perception information, object interaction relationships, and linguistic semantic constraints, thereby improving the adaptability of behavioral decisions to complex dynamic environments and the consistency of responses to semantic instructions.

[0057] In one embodiment, Figure 4 The flowchart illustrating spatiotemporal feature reasoning is shown, which may include the following steps: Step S401: The updated dynamic semantic behavior graph is transformed into structured graph data, and the global semantic features are mapped into semantic guidance vectors that represent instruction guidance.

[0058] Before using the dynamic semantic behavior graph for spatiotemporal feature inference, the updated dynamic semantic behavior graph undergoes data structure transformation. This transformation converts the spatial semantic features, edge relationships, and edge weights in the graph into a graph structure data representation to meet the input requirements of the subsequent graph structure computation process. The initial edge weights serve as weight parameters for the connection relationships between nodes, participating in the information transfer between them.

[0059] Simultaneously, the global semantic features are aligned with the spatial semantic feature vectors in the same dimension, and a semantic guidance vector is generated through a mapping transformation. This mapping transformation is achieved by performing a linear transformation on the global semantic features to establish a correspondence between the global semantic features and the spatial semantic features of the nodes. This allows the semantic guidance vector to participate in the spatial semantic feature update process, guiding and constraining the reasoning direction.

[0060] Step S402: Initially fuse the structured graph data with the semantic guidance vector to obtain the initial features of the nodes.

[0061] After constructing the graph structure data and generating semantic guidance vectors, the spatial semantic features of nodes in the structured graph data are fused with the semantic guidance vectors to form initial node features for subsequent spatiotemporal feature inference. Specifically, the spatial semantic feature vector and the semantic guidance vector corresponding to each node are added element-wise, so that the fused initial node features simultaneously contain the spatial semantic information of the environment and the semantic constraint information of the language instructions.

[0062] Step S403: Based on the initial weights of the edges and the similarity of the spatial semantic features, and by introducing a semantic guidance vector to constrain the update direction of the spatial semantic features, the attention weights between each node are calculated using a multi-head attention mechanism.

[0063] Based on graph-structured data, this paper models the interaction between each node and its neighboring nodes according to the connection relationships between nodes. By introducing a multi-head attention mechanism, the relationships between nodes are modeled from multiple subspaces, and the weights between nodes under different attention heads are calculated to obtain the degree of node association from different semantic perspectives.

[0064] In the calculation of attention weights, the interaction strength between nodes and semantic consistency are fused into a model, enabling the attention weights to simultaneously reflect structural relationships and semantic constraints. Specifically, the initial edge weights characterize the importance of existing interactions between the vehicle and environmental objects, while the similarity of spatial semantic features between nodes represents the degree of semantic association between different nodes. Building upon this, a semantic guidance vector is introduced into the update process of spatial semantic features. By directionally modulating the spatial semantic features, the changing trends of the spatial semantic features are aligned with the behavioral intent expressed by the linguistic instructions. This allows the allocation of attention weights to not only reflect environmental interactions but also further embody the guiding role of semantic instructions.

[0065] Step S404: Use attention weights to perform weighted aggregation of the spatial semantic features of adjacent nodes, and combine them with the initial fusion features of the current node to realize the transmission of interactive information between nodes.

[0066] After obtaining the attention weights between each node, the spatial semantic features of each node's neighborhood are weighted and aggregated. The spatial semantic features of adjacent nodes are weighted and summed according to their corresponding attention weights, and then further fused with the initial fusion features of the current node, thereby completing the update of the spatial semantic features.

[0067] During feature update, a multi-head attention mechanism is used to weight and combine the spatial semantic features of the neighborhood, and linear transformation and nonlinear activation processing are applied in conjunction with the node's own features to enhance the expression of the spatial semantic features. This allows the updated spatial semantic features to comprehensively reflect neighborhood interaction information and semantic constraint information. The spatial semantic feature update process satisfies the following relationship:

[0068] in, To represent nodes The spatial semantic features updated in the current inference cycle; For nodes Initial fusion features in the current inference cycle; Represents nodes A set of neighboring nodes that are connected; For the number of heads; For the first Each attention node and Attention weights; Indicates the first The feature transformation weight matrix corresponding to each attention head is used to linearly map the semantic features of the neighborhood space, so that they are mapped to a unified feature space and then participate in weighted aggregation. The linear transformation weight matrix represents the node's own characteristics and is used to map the spatial semantic features of the node before the current update, so as to preserve the contribution of the node's own information in the update process. Indicates the bias term; This represents a non-linear activation function. In practical applications, and All of these are learnable parameters, which are updated during model training using the backpropagation algorithm.

[0069] This weighted aggregation process ensures that each node's updated feature representation not only includes its own spatial semantic information but also incorporates information from other nodes with which it interacts, thus enabling the propagation and integration of interactive information within the graph structure. The updated spatial semantic features simultaneously reflect environmental object attributes, inter-object interaction relationships, and linguistic semantic constraints, allowing for a comprehensive evaluation of the vehicle's behavior in its current environment and the output of corresponding behavioral strategies.

[0070] In one embodiment, the spatiotemporal feature reasoning of the interaction behavior relationship further includes: After completing the weighted aggregation of spatial semantic features in the current inference cycle, the spatial semantic features of the previous inference cycle are jointly processed with the current weighted aggregated spatial semantic features, and the joint features are gating and filtering in conjunction with the semantic guidance vector.

[0071] In practical applications, a gated recurrent unit (GRU) can be introduced as a memory update unit. The spatial semantic features of the previous inference cycle are used as historical state inputs, and the spatial semantic features of the current inference cycle are used as current inputs. At the same time, the semantic guidance vector is introduced into the gating calculation process. Through update gates and reset gates, the historical and current information are dynamically adjusted to achieve selective retention and updating of feature information.

[0072] The gating mechanism may include update gates and reset gates. The update gate controls the proportion of historical features retained in the current feature from the previous inference cycle, so as to achieve continuous memory of historical information; the reset gate controls the degree of participation of historical features in the calculation of the current feature, so as to adjust the influence of historical information on the update of the current feature.

[0073] During the gating computation process, historical and current features are weighted and combined through update and reset gates, and the combination results are constrained by semantic guidance vectors. This ensures that the retained information prioritizes the behavioral intent expressed by the language instructions, thereby effectively suppressing feature components that are inconsistent with the current semantic constraints while integrating historical interaction information and current environmental information.

[0074] After completing the gating update, the filtered feature results are used as the final feature representation of the current inference cycle. The spatial semantic features of the current inference cycle are updated by memory, so that the spatial semantic features can retain historical interaction information during continuous inference and be dynamically adjusted in combination with the current environmental state and semantic constraints, thereby improving the stability and continuity of spatiotemporal feature inference.

[0075] Step S106: Based on the risk distribution information of the behavioral strategy and environmental objects, generate a target driving trajectory that meets the semantic constraints.

[0076] After obtaining the vehicle's behavior strategy and the risk distribution information of environmental objects, the behavior strategy is used as the behavioral constraint for trajectory generation, and the risk distribution information is used as the basis for risk assessment in the trajectory space. Constraint modeling is then applied to the trajectory generation process to generate a target driving trajectory that satisfies semantic constraints. The target driving trajectory can be represented as:

[0077] In the above formula, Indicates the vehicle's time The corresponding two-dimensional spatial coordinates; Indicates the vehicle's time The heading angle is used to characterize the vehicle's direction of travel; It represents a continuous trajectory sequence consisting of multiple discrete time-state points.

[0078] During trajectory generation, behavioral strategies are used to define the overall behavioral trend and movement pattern of the trajectory, while risk distribution information is used to characterize potential hazardous areas in the environment and constrain the spatial distribution range of the trajectory. By constraining and iteratively optimizing the trajectory parameters while satisfying vehicle dynamics constraints, the generated target driving trajectory ensures driving safety, conforms to the behavioral intent expressed by the language command, and can adapt to the dynamic changes of the environment.

[0079] In one embodiment, the risk distribution information of environmental objects is obtained in the following manner: Preset risk weights are assigned based on the category of environmental objects, with different categories corresponding to different risk levels. For example, the risk weight for an environmental object of pedestrian type can be set to 1.0; for an environmental object of vehicle type, the risk weight can be set to 0.7; and for static obstacles, the risk weight can be set to 0.5, thus forming a risk weight mapping relationship based on object category. The above values ​​are for illustrative purposes only and can be adjusted according to road scenarios and safety requirements in practical applications. The risk weight characterizes the degree of impact of different types of environmental objects on vehicle driving safety. The higher the risk weight, the higher the potential threat level of that type of object to vehicle driving, and the higher the corresponding avoidance priority during trajectory planning.

[0080] Based on the distances between the vehicle and various environmental objects and their corresponding risk weights, an exponential decay function is used to calculate the risk score of each environmental object to the spatial location. The risk score quantifies the overall degree of danger at any given location in space under the combined influence of multiple environmental objects. The semantic risk field at a given location satisfies the following relationship:

[0081] in, Indicates spatial location The overall risk score at the location, Indicates vehicle and the The distance between environment objects This represents the corresponding semantic risk weight. Parameters used to control the attenuation range.

[0082] By spatially superimposing the risk scores of each environmental object, a continuously distributed semantic risk field is formed to characterize potential dangerous areas in the environment and provide a risk constraint basis for subsequent trajectory generation.

[0083] In one embodiment, Figure 5 A schematic diagram of the process for generating the target driving trajectory may include the following steps: Step S501: Based on the semantic risk field, perform trajectory space sampling to generate a set of candidate driving trajectories.

[0084] Starting from the current state of the vehicle, under the condition of satisfying the vehicle kinematic constraints, the trajectory space is discretely sampled, and multiple continuous trajectory curves are generated by polynomial interpolation, thus forming a set of candidate driving trajectories.

[0085] Introducing a semantic risk field during trajectory sampling As a spatial constraint, the risk score of the area traversed by the trajectory is evaluated during the sampling process, so that the sampled trajectory tends to stay away from high-risk areas from the generation stage. Each generated candidate trajectory corresponds to a set of trajectory parameters, including position sequence, velocity change, and turning change, which are used for subsequent screening and optimization.

[0086] Step S502: Select candidate trajectories from the candidate driving trajectory set that avoid areas with risk scores higher than a preset threshold as the initial driving trajectory.

[0087] Risk assessment is performed on each trajectory in the candidate trajectory set by calculating the maximum risk score or cumulative risk score of the region traversed by the trajectory in the semantic risk field and comparing it with a preset risk threshold.

[0088] Trajectories that meet the risk constraints (i.e., the risk score of the area traversed by the trajectory is below the threshold) are retained, and trajectories that meet the constraints of vehicle driving continuity and behavioral strategy are selected as the initial driving trajectory.

[0089] This screening process ensures that the initial trajectory has basic safety during the generation stage, providing a reasonable initial solution for subsequent optimization.

[0090] Step S503: Construct an objective function that includes smoothness loss, risk field penalty loss, and behavioral policy loss, and solve the objective function based on a constrained optimization algorithm to obtain the target driving trajectory.

[0091] The trajectory optimization problem is formalized as a constrained objective function optimization problem, where the objective function is expressed as:

[0092] In the above formula, The smoothness loss used to characterize the trajectory; Used to characterize the risk penalty of a trajectory in a semantic risk field; Used to characterize the degree of deviation of the trajectory from the constraints of the behavioral strategy; , and These are the adjustment weights for each loss term in the objective function, used to allocate the importance of different optimization objectives.

[0093] Based on the objective function, a sequential quadratic programming method is used to iteratively optimize the trajectory parameters. Under the premise of satisfying the vehicle dynamics constraints, the trajectory is gradually adjusted to make the objective function converge, thereby obtaining the target driving trajectory that satisfies multiple constraints.

[0094] In one embodiment, the process of solving the objective function based on the constraint optimization algorithm includes iteratively optimizing and adjusting the initial driving trajectory to obtain a target driving trajectory that satisfies multiple constraint conditions.

[0095] Specifically, a state description for trajectory optimization is constructed using the trajectory parameters of the initial driving trajectory, the risk score output by the semantic risk field, the position change information of environmental objects, and the constraint deviation of the behavioral strategy, forming a state space to characterize the current trajectory state. The trajectory parameters include the position, velocity, and steering angle information corresponding to each discrete moment in the trajectory; the position change information of environmental objects is used to characterize the dynamic changes in the environment; and the constraint deviation of the behavioral strategy is used to characterize the degree to which the current trajectory meets the semantic behavioral requirements.

[0096] Based on this, the adjustment amount of trajectory parameters is used as the action description in the trajectory optimization process, forming an action space. The action space is used to represent the set of executable operations for adjusting the trajectory, including the adjustment amount of the position offset, speed change, and steering angle change of the trajectory points. All adjustment amounts are limited within the vehicle dynamics constraints to ensure that the adjusted trajectory is executable.

[0097] In the trajectory optimization process, the trajectory is iteratively solved based on the objective function. After each trajectory adjustment, the current trajectory is evaluated based on the risk feedback in the semantic risk field and the satisfaction of behavioral constraints, thereby constructing a reward function to guide the direction of trajectory adjustment. The reward function can be specifically expressed as:

[0098] in, This indicates the evaluation result of the current trajectory; This represents the objective function value, used to reflect the overall optimization degree of the trajectory in terms of smoothness, safety, and semantic consistency; This represents the maximum risk score within the area traversed by the trajectory, used to characterize the least safe level of the trajectory; and is a weighting coefficient used to adjust the ratio of influence between the degree of optimization of the objective function and the risk constraints.

[0099] In the aforementioned optimization process, the initial driving trajectory is progressively corrected through continuous perception of the state space, iterative adjustment of the action space, and feedback evaluation based on the reward function, causing the trajectory to continuously approach the optimal solution of the objective function during the optimization process. Simultaneously, by introducing an iterative adjustment mechanism based on reward feedback, local optima that may occur during the constraint optimization process are corrected, further enhancing the trajectory's adaptability to dynamic environmental changes while satisfying vehicle dynamics and risk constraints, ultimately generating a target driving trajectory that meets semantic constraints.

[0100] By employing the aforementioned technical solution, based on acquiring visual data of the vehicle's surrounding environment and identifying environmental objects, a dynamic semantic behavior graph is constructed. This graph structurally represents environmental objects and their interaction relationships. Furthermore, by combining global semantic features obtained from parsing language commands, the dynamic semantic behavior graph is updated, ensuring that the environmental understanding process is constrained by both perceptual and semantic information. Building upon this foundation, spatiotemporal feature reasoning of interactive behavior relationships allows for a comprehensive modeling of the dynamic interaction between the vehicle and environmental objects, thereby generating behavioral strategies that align with semantic intent. Further, by incorporating risk distribution information of environmental objects, the trajectory generation process is constrained and modeled. This ensures that the generated target driving trajectory satisfies vehicle dynamics constraints while effectively avoiding high-risk areas and maintaining consistency with the behavioral intent expressed by the language commands. Consequently, the vehicle exhibits higher decision-making rationality, driving safety, and semantic consistency in complex dynamic environments, thereby improving the overall performance of the trajectory generation results.

[0101] In one embodiment, a vehicle control method is provided, which generates a target driving trajectory for the vehicle using the aforementioned vehicle trajectory generation method, and controls the vehicle based on the target driving trajectory. Based on information about the vehicle's surrounding environment, semantic language commands, and risk distribution information of environmental objects, a target driving trajectory that satisfies semantic constraints is generated. The target driving trajectory is a trajectory sequence describing the vehicle's motion state over a future period. Specific limitations on the target driving trajectory can be found in the description of the vehicle trajectory generation method above, and will not be repeated here.

[0102] After obtaining the target driving trajectory, the trajectory is parsed and processed to convert it into vehicle control commands, including speed control and steering angle control. These speed and steering angle control commands are then sent to the vehicle control system for execution, causing the vehicle to travel along the target trajectory.

[0103] The above method converts the trajectory generation result into control commands that can be executed by the vehicle, and drives the vehicle to complete the corresponding driving control. This enables the vehicle to run stably along the target driving trajectory that meets the semantic constraints, ensuring driving safety while improving the vehicle's adaptability to complex environmental changes and the stability of trajectory tracking.

[0104] like Figure 6The diagram illustrates a vehicle data processing procedure. In the perception and analysis phase, visual data of the vehicle's surrounding environment is acquired. Object recognition is performed on the visual data to extract the spatial state and category attributes of environmental objects, forming spatial semantic features. Simultaneously, externally input voice commands are received, and a language semantic parsing module extracts global semantic features. These global semantic features represent the semantic intent information corresponding to the voice command. Subsequently, the object semantic information and the global semantic features are cross-modal aligned to generate initial cross-modal fusion features that highly couple the command intent with the environmental context.

[0105] In the graph modeling and risk perception phase, a dynamic semantic behavior graph is constructed based on the aforementioned spatial semantic features. This graph uses environmental objects as nodes and the spatial locations and interaction logic between objects as edges. Simultaneously, semantic risk field modeling is carried out using spatial semantic features. Through preset risk weights and distance decay functions, a semantic risk field characterizing the distribution of environmental hazard levels is constructed.

[0106] During the reasoning and decision-making phase, nodes in the dynamic semantic behavior graph are updated. This process maps global semantic features to obtain semantic guidance vectors, which serve as semantic constraint inputs in the graph reasoning process. Based on the behavioral relationship reasoning of a graph neural network (GNN), the spatiotemporal interaction features between nodes are deeply mined. In this phase, the GNN utilizes multi-head attention mechanisms and gating filtering, combined with initial fused features and node update states, to infer the vehicle's behavioral strategy in the current semantic environment.

[0107] During the planning and execution phase, the behavioral strategy and semantic risk field are jointly input. A hybrid optimization algorithm combining sequential quadratic programming and reinforcement learning is used to optimize the vehicle's trajectory under constraints, generating a candidate trajectory solution set while satisfying both semantic risk constraints and vehicle dynamics constraints. Sequential quadratic programming is responsible for finding the mathematically optimal trajectory that meets physical characteristic limitations under the constraints of the semantic risk field, while reinforcement learning iteratively fine-tunes the trajectory parameters based on the behavioral strategy to generate the target trajectory that satisfies the semantic constraints. Finally, this target trajectory is sent to the vehicle's underlying control and execution module to drive the vehicle to complete the corresponding driving actions.

[0108] In one embodiment, a vehicle is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement a vehicle trajectory generation method or a vehicle control method.

[0109] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0110] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0111] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for generating a vehicle driving trajectory, characterized in that, include: Acquire visual data of the vehicle's surrounding environment and perform environmental object recognition on the visual data; A dynamic semantic behavior graph is constructed based on the identified environmental objects. The dynamic semantic behavior graph includes nodes and edges, where each node corresponds one-to-one with an environmental object, and the edges represent the interactive behavior relationships between the vehicle and each environmental object. Each environmental object is mapped to a unique node in the dynamic semantic behavior graph, and the spatial state information and semantic feature vector of each environmental object are used as attributes of the corresponding node. Based on the relative motion state and semantic feature similarity between the vehicle and each node, directed edges are established between the vehicle and each node through a combination of rule matching and similarity calculation, and each edge is assigned an initial weight and behavioral relationship. The behavioral relationships include avoidance, following, yielding, and stopping. Obtain externally input language instructions, perform semantic parsing on the language instructions, and extract global semantic features; update the dynamic semantic behavior graph using the global semantic features; Based on the global semantic features and the updated dynamic semantic behavior graph, spatiotemporal feature reasoning is performed on the interaction behavior relationship to obtain the vehicle's behavior strategy. Based on the behavioral strategy and the risk distribution information of the environmental objects, a target driving trajectory that satisfies semantic constraints is generated; wherein, the vehicle is a low-speed autonomous driving vehicle operating within a limited area, and the vehicle's driving speed is below a preset threshold; the risk distribution information of the environmental objects includes a semantic risk field; when generating the target driving trajectory, the behavioral strategy and the semantic risk field are jointly input, and a hybrid optimization algorithm combining sequential quadratic programming and reinforcement learning is used to optimize the vehicle's driving trajectory under constraints, wherein the sequential quadratic programming is used to solve for the mathematically optimal trajectory that satisfies physical characteristic constraints under the constraints of the semantic risk field, and the reinforcement learning is used to iteratively fine-tune the trajectory parameters based on the behavioral strategy.

2. The vehicle trajectory generation method according to claim 1, characterized in that, Updating the dynamic semantic behavior graph using the global semantic features includes: A cross-attention mechanism is employed, using the global semantic features as the query and the spatial semantic features of each node in the dynamic semantic behavior graph as the key and value, to calculate the attention weight; the spatial semantic features include spatial state information and semantic feature vectors. The spatial semantic features of each node are updated by weighting according to the attention weights to obtain the updated spatial semantic features, thus completing the update of the dynamic semantic behavior graph.

3. The vehicle trajectory generation method according to claim 2, characterized in that, The spatiotemporal feature reasoning of the interaction behavior relationship includes: The updated dynamic semantic behavior graph is transformed into structured graph data, and the global semantic features are mapped into semantic guidance vectors representing instruction guidance. The structured graph data and the semantic guidance vector are initially fused to obtain the initial fusion features of the nodes; based on the similarity between the initial weights of the edges and the spatial semantic features, the semantic guidance vector is introduced to the spatial semantic features. The update direction of semantic features is constrained, and the attention weights between nodes are calculated using a multi-head attention mechanism; The attention weights are used to aggregate the spatial semantic features of adjacent nodes in a weighted manner, and combined with the initial fusion features of the current node, to realize the transmission of interactive information between nodes.

4. The vehicle trajectory generation method according to claim 3, characterized in that, The spatiotemporal feature reasoning of the interaction behavior relationship further includes: The spatial semantic features of the previous inference cycle are jointly processed with the current weighted aggregated spatial semantic features, and gating is performed based on the semantic guidance vector to retain feature components consistent with the language instructions and suppress irrelevant feature components. The spatial semantic features of the current inference cycle are updated based on the filtered feature results.

5. The vehicle trajectory generation method according to claim 1, characterized in that, The risk distribution information of the environmental objects is obtained in the following way: Assign preset risk weights based on the category of the environmental object; The risk score is calculated using an exponential decay function based on the distance between the vehicle and the environmental object and the risk weight. By summarizing the risk scores of each environmental object, a semantic risk field representing the distribution of environmental hazard levels is constructed.

6. The vehicle trajectory generation method according to claim 5, characterized in that, The generation of the target driving trajectory that satisfies the semantic constraints includes: Based on the semantic risk field, trajectory space sampling is performed to generate a set of candidate driving trajectories; Select candidate trajectories from the candidate driving trajectory set that avoid areas with risk scores higher than a preset threshold as the initial driving trajectory; An objective function is constructed that includes smoothness loss, risk field penalty loss, and behavioral policy loss. The objective function is then solved using a constrained optimization algorithm to obtain the target driving trajectory.

7. The vehicle trajectory generation method according to claim 6, characterized in that, Solving the objective function based on the constraint optimization algorithm includes: Using the trajectory parameters of the initial driving trajectory, the risk score output by the semantic risk field, the position change information of the environmental object, and the constraint deviation of the behavior strategy as the state space, and the adjustment amount of the trajectory parameters as the action space, the initial driving trajectory is adjusted by iteratively optimizing the objective function to generate the target driving trajectory; the adjustment amount includes at least one change in trajectory point position, speed, and steering angle.

8. A vehicle control method, characterized in that, include: The vehicle trajectory generation method as described in any one of claims 1 to 7 is used to generate the target trajectory of the vehicle. Control the vehicle to travel along the target trajectory.

9. A vehicle, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 8.

Images