Vehicle planning driving track generation method and device, vehicle, and storage medium

Abstract

The application relates to a vehicle planning driving track generation method and device, a vehicle and a storage medium, and particularly relates to the technical field of vehicle automatic driving. The method obtains surrounding image data of a vehicle, vehicle state data and networked perception and cooperation data of other traffic participants, determines an image other traffic participant key-value matrix and a road topology key-value matrix based on the surrounding image data, constructs an other traffic participant spatial position query matrix in combination with the vehicle state data and the networked perception and cooperation data, fuses to obtain an other traffic participant fusion representation feature key-value matrix, and generates a vehicle planning driving track in combination with a preset vehicle planning driving track query vector. The surrounding image data and the networked perception and cooperation data are fused, traffic participants in a visual blind area and an over-the-horizon area are completely covered, vehicle track planning conflicts and collision risks caused by the visual blind area and the over-the-horizon area are avoided, and the safety of a vehicle planning track in a visual blind area and an over-the-horizon complex traffic scene is improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus, vehicle, and storage medium for generating a planned driving trajectory for a vehicle. Background Technology

[0002] With the development and increasing popularity of autonomous driving technology, vehicle trajectory planning is a core component of autonomous driving. Currently, the mainstream method for generating vehicle trajectory planning relies on onboard vision sensors to collect environmental images, extract image features from these images, and then perform spatial transformation and perceptual reasoning based on the extracted features to generate the vehicle's planned trajectory.

[0003] However, generating vehicle trajectory plans based on environmental images can introduce collision risks. For example, vehicle cameras are limited by physical line-of-sight and obstacle obstruction, which can create blind spots in the environmental images. Consequently, generating vehicle trajectory plans based on environmental images can lead to collision risks. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this application provides a method, apparatus, vehicle and storage medium for generating vehicle planning driving trajectory.

[0005] In a first aspect, this application provides a method for generating a vehicle's planned driving trajectory, characterized by comprising: Acquire surrounding image data, vehicle status data, and networked perception and collaborative data from other traffic participants; Based on the surrounding image data, determine the key-value matrix of other traffic participants in the image and the key-value matrix of the road topology; Based on the vehicle status data and the network-connected sensing collaborative data, determine the spatial location query matrix of other traffic participants; Based on the image other traffic participant key value matrix, the road topology key value matrix, and the other traffic participant spatial location query matrix, determine the other traffic participant fusion representation feature key value matrix; The vehicle's planned driving trajectory is determined based on the fusion representation feature key matrix of other traffic participants, the image key matrix of other traffic participants, the road topology key matrix, and the preset vehicle planned driving trajectory query vector.

[0006] In an optional implementation, determining the key-value matrix of other traffic participants and the road topology key-value matrix based on the surrounding image data includes: Other traffic participant features and road topology features are extracted from the surrounding image data to obtain other traffic participant matrix and road topology matrix; The other traffic participant matrix and the road topology matrix are projected to obtain the image other traffic participant key matrix and the road topology key matrix, respectively.

[0007] In an optional implementation, determining the spatial location query matrix of other traffic participants based on the vehicle status data and the network-connected sensing collaborative data includes: The network-connected sensing collaborative data is subjected to dual-scale normalization processing to obtain a normalized network-connected sensing collaborative matrix; Based on the position coordinates in the vehicle status data and the position coordinates in the connected sensing and coordination data, the relative position coordinates of other traffic participants relative to the vehicle are determined. Based on the relative position coordinates of the other traffic participants with respect to the vehicle, determine the spatial position matrix of the other traffic participants; Based on the normalized networked sensing collaboration matrix and the spatial location matrix of other traffic participants, the spatial location query matrix of other traffic participants is obtained.

[0008] In an optional implementation, the step of performing dual-scale normalization processing on the connected sensing collaborative data to obtain a normalized connected sensing collaborative matrix includes: Determine the collaborative scale corresponding to the network-connected sensing collaborative data, and the visual scale corresponding to the surrounding image data; The data mapping ratio is determined based on the collaborative scale and the visual scale; The network-connected sensing collaborative data is scale-mapped according to the mapping ratio to obtain a normalized network-connected sensing collaborative matrix.

[0009] In an optional implementation, obtaining the spatial location query matrix of other traffic participants based on the normalized networked sensing cooperation matrix and the spatial location matrix of other traffic participants includes: The heading angle in the normalized network sensing cooperative matrix is ​​transformed by sine and cosine to obtain a unit vector of heading angle. By fusing the position coordinates, velocity, and heading angle unit vector of the normalized network sensing cooperative matrix, a highly sensitive implicit feature matrix is ​​obtained; The acceleration data in the normalized network sensing cooperative matrix is ​​processed to obtain a low-sensitivity acceleration matrix; By fusing the highly sensitive implicit feature matrix with the low-sensitivity acceleration matrix, the spatial location implicit feature matrix of other traffic participants is obtained; By fusing the implicit feature matrix of the spatial location of the other traffic participants with the spatial location matrix of the other traffic participants, a spatial location query matrix of the other traffic participants is obtained.

[0010] In an optional implementation, the step of determining the fusion representation feature key matrix of other traffic participants based on the image other traffic participant key matrix, the road topology key matrix, and the spatial location query matrix of other traffic participants includes: Using the spatial location query matrix of other traffic participants as the query basis, the image other traffic participant key matrix as the key, and the image other traffic participant value matrix as the value, an interactive query is performed to obtain the implicit query matrix of spatial location of other traffic participants; Using the implicit query matrix of spatial location of other traffic participants as the query basis, the road topology key matrix as the key, and the road topology value matrix as the value, an interactive query is carried out to obtain the fusion representation feature matrix of other traffic participants; The fusion representation feature matrix of the other traffic participants is projected to obtain the fusion representation feature key value matrix of the other traffic participants.

[0011] In an optional implementation, determining the vehicle's planned driving trajectory based on the fusion representation feature key-value matrix of the other traffic participants, the image key-value matrix of the other traffic participants, the road topology key-value matrix, and a preset vehicle planned driving trajectory query vector includes: Using the preset autonomous vehicle planning trajectory query vector as the query basis, the fusion representation feature key matrix of other traffic participants as the key, and the fusion representation feature value matrix of other traffic participants as the value, an interactive query is performed to obtain the first autonomous vehicle planning trajectory decision feature matrix. Using the first autonomous vehicle planning and driving trajectory decision feature matrix as the query basis, the image other traffic participant key matrix as the key, and the image other traffic participant value matrix as the value, an interactive query is performed to obtain the second autonomous vehicle planning and driving trajectory decision feature matrix. Using the second autonomous vehicle planning and driving trajectory decision feature matrix as the query basis, the road topology key-value matrix as the key, and the road topology value matrix as the value, an interactive query is performed to obtain the third autonomous vehicle planning and driving trajectory decision feature matrix; The vehicle's planned driving trajectory is determined based on the first vehicle planning driving trajectory decision feature matrix, the second vehicle planning driving trajectory decision feature matrix, and the third vehicle planning driving trajectory decision feature matrix.

[0012] In an optional implementation, determining the vehicle's planned driving trajectory based on the first vehicle planning driving trajectory decision feature matrix, the second vehicle planning driving trajectory decision feature matrix, and the third vehicle planning driving trajectory decision feature matrix includes: By fusing the first autonomous vehicle planning driving trajectory decision feature matrix, the second autonomous vehicle planning driving trajectory decision feature matrix, and the third autonomous vehicle planning driving trajectory decision feature matrix, a vehicle fusion planning driving trajectory decision feature matrix is ​​obtained. The dimensionality reduction process is performed on the vehicle fusion planning driving trajectory decision feature matrix to obtain the set of relative position coordinates of the vehicle in future time periods; The vehicle's planned driving trajectory is obtained by integrating the set of relative position coordinates of the vehicle in the future time period.

[0013] Secondly, this application provides a vehicle planning and driving trajectory generation device, comprising: The data acquisition module is used to acquire surrounding image data, vehicle status data, and network-connected perception and collaborative data from other traffic participants. The first key value matrix determination module is used to determine the key value matrix of other traffic participants and the road topology key value matrix based on the surrounding image data. The spatial location query matrix determination module is used to determine the spatial location query matrix of other traffic participants based on the vehicle status data and the network-connected sensing collaborative data. The second key value matrix determination module is used to determine the fusion representation feature key value matrix of other traffic participants based on the key value matrix of other traffic participants in the image, the road topology key value matrix, and the spatial location query matrix of other traffic participants; The vehicle planning and driving trajectory generation module is used to determine the vehicle's planned driving trajectory based on the fusion representation feature key value matrix of other traffic participants, the image other traffic participant key value matrix, the road topology key value matrix, and the preset vehicle planning and driving trajectory query vector.

[0014] Thirdly, this application provides a vehicle including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the vehicle planning and driving trajectory generation method described in any of the first aspects.

[0015] Fourthly, this application provides a computer-readable storage medium storing a program for a vehicle planning and driving trajectory generation method, wherein when the program for the vehicle planning and driving trajectory generation method is executed by a processor, it implements the steps of the vehicle planning and driving trajectory generation method described in any of the first aspects.

[0016] The beneficial effects of this application are: By acquiring network-connected perception collaborative data of other traffic participants, a spatial location query matrix of other traffic participants that can characterize the spatial location of targets in blind spots is generated. This other traffic participant spatial location query matrix is ​​then fused with the image other traffic participant key-value matrix obtained from the vehicle's surrounding image data, as well as the road topology key-value matrix, to form a fused representation feature key-value matrix of other traffic participants. This ensures that the input for generating the driving trajectory includes not only traffic participants visible within the vehicle's camera's line of sight, but also data of other traffic participants in visual blind spots due to physical occlusion or beyond line of sight. Based on the fused representation feature key-value matrix of other traffic participants, and combined with a preset vehicle-planned driving trajectory query vector, the driving trajectory is generated. This avoids trajectory conflicts caused by missed detection of traffic participants due to image visual blind spots, thus solving the collision risk problem of simply relying on environmental images to generate trajectories and improving vehicle driving safety in occluded and beyond-line-of-sight scenarios. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a method for generating a vehicle's planned driving trajectory, as provided in this application embodiment; Figure 2 A flowchart illustrating a method for determining the key-value matrix of other traffic participants and the road topology key-value matrix in an image, provided in this application embodiment; Figure 3 A flowchart illustrating a method for determining the spatial location query matrix of other traffic participants, provided in an embodiment of this application; Figure 4 A flowchart illustrating a method for determining the fusion representation feature key-value matrix of other traffic participants, provided in an embodiment of this application; Figure 5A flowchart illustrating a method for determining the planned driving trajectory of a vehicle, as provided in this application embodiment; Figure 6 This is a schematic diagram of the structure of a vehicle planning and driving trajectory generation device provided in an embodiment of this application; Figure 7 This is a structural diagram of a vehicle provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] like Figure 1 The diagram shown is a schematic representation of the implementation process of a vehicle planning and driving trajectory generation method provided in this application embodiment. This method is applied to vehicles and specifically includes the following steps: S101 acquires surrounding image data, vehicle status data, and networked perception and collaborative data from other traffic participants.

[0022] In this embodiment of the application, the vehicle is equipped with an image acquisition device (e.g., an in-vehicle camera) and a communication device (e.g., a C...). (V2X communication components). The vehicle acquires surrounding image data through image acquisition devices; acquires vehicle status data through the on-board bus; and acquires networked perception and coordination data of other traffic participants through communication devices. Among them, the vehicle status data includes at least the vehicle's position coordinates; the networked perception and coordination data includes at least the position coordinates, vehicle speed, acceleration, and heading angle of other traffic participants; it can obtain networked perception and coordination data of other traffic participants beyond line of sight and in blind spots, providing complete raw data support for subsequent processing.

[0023] In this embodiment, other traffic participants include communication devices (e.g., C-V2X communication components). These other traffic participants encapsulate their own position coordinates, vehicle speed, acceleration, and heading angle into V2X data. This V2X data is then serialized using Protobuf to obtain serialized V2X data, which is broadcast outwards via the communication devices. The vehicle receives the serialized V2X data, and after deserialization using Protobuf, obtains the network-connected sensing and collaborative data of the other traffic participants.

[0024] S102, based on surrounding image data, determine the key-value matrix of other traffic participants in the image and the key-value matrix of road topology.

[0025] In this embodiment, based on surrounding image data, a key-value matrix of other traffic participants and a road topology key-value matrix are determined. This provides network-side spatial location input for subsequent fusion based on the spatial location query matrix of other traffic participants.

[0026] S103, based on vehicle status data and network-connected sensing collaborative data, determines the spatial location query matrix of other traffic participants.

[0027] In this embodiment of the application, a spatial location query matrix for other traffic participants is determined based on vehicle status data and network-connected sensing collaborative data; such that the input used to generate the driving trajectory includes data of other traffic participants who are in visual blind spots due to physical obstruction or beyond line of sight.

[0028] S104. Based on the key value matrix of other traffic participants in the image, the key value matrix of road topology, and the spatial location query matrix of other traffic participants, determine the key value matrix of fused representation features of other traffic participants.

[0029] In this embodiment, based on the image other traffic participant key value matrix, road topology key value matrix, and other traffic participant spatial location query matrix, the fusion representation feature key value matrix of other traffic participants is determined; this avoids trajectory conflicts caused by missed detection of traffic participants due to image visual blind spots, and improves vehicle driving safety in occluded and beyond-line-of-sight scenarios.

[0030] S105. Based on the key-value matrix of the fusion representation features of other traffic participants, the key-value matrix of other traffic participants in the image, the key-value matrix of the road topology, and the preset query vector of the self-planned driving trajectory, determine the planned driving trajectory of the self-driving vehicle.

[0031] In this embodiment, the vehicle's planned driving trajectory is determined based on the key-value matrix of other traffic participants' fusion representation features, the key-value matrix of other traffic participants in the image, the key-value matrix of road topology, and the preset query vector of the vehicle's planned driving trajectory.

[0032] This application embodiment acquires network-connected perception collaborative data of other traffic participants to generate a spatial location query matrix of other traffic participants that can characterize the spatial location of targets in blind spots. This other traffic participant spatial location query matrix is ​​then fused with an image key-value matrix of other traffic participants obtained from image data surrounding the vehicle, and a road topology key-value matrix to form a fused representation feature key-value matrix of other traffic participants. This ensures that the input for generating the driving trajectory includes not only traffic participants visible within the vehicle's camera's line of sight, but also data of other traffic participants in visual blind spots due to physical occlusion or beyond line-of-sight. Based on the fused representation feature key-value matrix of other traffic participants, a driving trajectory is generated by combining a preset vehicle-planned driving trajectory query vector. This avoids trajectory conflicts caused by missed detection of traffic participants due to image visual blind spots, thus solving the collision risk problem associated with solely relying on environmental images to generate trajectories and improving vehicle driving safety in occluded and beyond-line-of-sight scenarios.

[0033] In the embodiments of this application, such as Figure 2 As shown, the specific execution steps of "determining the key-value matrix of other traffic participants and the road topology key-value matrix based on surrounding image data" in S102 are as follows: S201, extract features of other traffic participants and road topology features from surrounding image data to obtain the other traffic participant matrix and road topology matrix.

[0034] In this embodiment, a pre-trained vehicle trajectory planning output model is obtained. Surrounding image data, vehicle state data, and network-connected perception collaborative data of other traffic participants are input into the pre-trained vehicle trajectory planning output model. The pre-trained vehicle trajectory planning output model extracts features of other traffic participants and road topology features from the surrounding image data to obtain a matrix of other traffic participants and a road topology matrix.

[0035] Specifically, the pre-trained vehicle trajectory planning output model includes an image processing module, which comprises a ResNet-50 backbone network, a Feature Pyramid Network (FPN), and a BEVFormer network. The surrounding image data has a dimension of H×W×3. The ResNet-50 backbone network extracts features of other traffic participants and road topology features from the surrounding image data. The FPN then fuses these features to obtain a two-dimensional image feature map with dimensions of H'×W'×C. Finally, the BEVFormer network performs BEV space transformation on the two-dimensional image feature map to obtain the other traffic participant matrix and the road topology matrix.

[0036] S202, the other traffic participant matrix and the road topology matrix are projected to obtain the image other traffic participant key matrix and the image other traffic participant value matrix.

[0037] In this embodiment, the pre-trained vehicle trajectory planning output model projects the other traffic participant matrix and the road topology matrix to obtain the image other traffic participant key-value matrix and the road topology key-value matrix. The image other traffic participant key-value matrix includes the image other traffic participant key matrix and the image other traffic participant value matrix, and the road topology key-value matrix includes the road topology key matrix and the road topology value matrix.

[0038] Specifically, the image processing module includes a traffic participant projection layer and a road topology projection layer; the traffic participant projection layer includes a traffic participant key projection layer and a traffic participant value projection layer; the road topology projection layer includes a road topology key projection layer and a road topology value projection layer; the image other traffic participant key matrix is ​​obtained by projecting the other traffic participant matrix through the traffic participant key projection layer, and the image other traffic participant value matrix is ​​obtained by projecting the other traffic participant matrix through the traffic participant value projection layer; the road topology key matrix is ​​obtained by projecting the road topology matrix through the road topology key projection layer, and the road topology value matrix is ​​obtained by projecting the road topology matrix through the road topology value projection layer.

[0039] like Figure 3 As shown, the specific steps in S103, "determining the spatial location query matrix of other traffic participants based on vehicle status data and network-connected sensing collaborative data," are as follows: S301, perform dual-scale normalization processing on the network-connected sensing collaborative data to obtain the normalized network-connected sensing collaborative matrix.

[0040] In this embodiment, the process of performing dual-scale normalization on the network-connected sensing collaborative data to obtain a normalized network-connected sensing collaborative matrix involves: determining the collaborative scale corresponding to the network-connected sensing collaborative data and the visual scale corresponding to the surrounding image data; determining the data mapping ratio based on the collaborative scale and the visual scale; and performing scale mapping processing on the network-connected sensing collaborative data according to the mapping ratio to obtain the normalized network-connected sensing collaborative matrix.

[0041] Specifically, a pre-trained vehicle trajectory planning output model is configured with a normalization processing module. This module determines the collaborative scale corresponding to the connected sensing collaborative data and the visual scale corresponding to the surrounding image data; the collaborative scale is larger than the visual scale. The visual scale is divided by the collaborative scale to obtain the data mapping ratio. The connected sensing collaborative data is mapped according to the data mapping ratio (during the mapping process, only the position coordinates in the connected sensing collaborative data are compressed according to the data mapping ratio, while the other data corresponding to the position coordinates remain unchanged, such as acceleration and heading angle). The connected sensing collaborative data is processed to be at the same scale as the surrounding image data, retaining the connected sensing collaborative data beyond the visual scale range to compensate for the visual blind spots in the surrounding image data. This completes the scale alignment between the connected sensing collaborative data and the surrounding image data, resulting in a normalized connected sensing collaborative matrix. This avoids directly truncating the connected sensing collaborative data beyond line of sight, and while unifying the data scale, it fully preserves the blind spots and other traffic participants beyond line of sight, providing a data foundation for the subsequent generation of spatial location query matrices for other traffic participants.

[0042] S302, based on the position coordinates in the vehicle status data and the position coordinates in the networked sensing and coordination data, determines the relative position coordinates of other traffic participants relative to the vehicle.

[0043] In this embodiment of the application, the specific process of determining the relative position coordinates of other traffic participants relative to the vehicle based on the position coordinates in the vehicle state data and the position coordinates in the networked sensing collaborative data is as follows: The pre-trained vehicle planning trajectory output model is configured with a spatial position matrix processing module. Through the spatial position matrix processing module, the position coordinates in the vehicle state data (vehicle position coordinates) and the position coordinates in the networked sensing collaborative data (other traffic participant position coordinates) are subtracted to obtain the real-time two-dimensional spatial relative coordinate vectors (Δx, Δy) of other traffic participants relative to the vehicle.

[0044] S303, determine the spatial position matrix of other traffic participants based on their relative position coordinates to the vehicle.

[0045] In this embodiment, the specific process of determining the spatial position matrix of other traffic participants based on their relative position coordinates to the vehicle is as follows: The spatial position matrix processing module divides the channel into a first sub-channel and a second sub-channel, with the first sub-channel corresponding to the horizontal axis and the second sub-channel corresponding to the vertical axis; the first sub-channel uses sine and cosine to process the relative horizontal coordinates in the relative position coordinates to obtain the horizontal coordinate code, and the second sub-channel uses sine and cosine to process the relative vertical coordinates in the relative position coordinates to obtain the vertical coordinate code. All (horizontal coordinate code and vertical coordinate code) are combined to form the spatial position matrix of other traffic participants.

[0046] For example, the spatial position matrix processing module divides the channel dimension C into two sub-channel groups, each with a dimension of C / 2, corresponding to the x-axis and y-axis of the relative coordinates, respectively. For the x-axis coordinate value Δx, at the i-th channel position in the corresponding sub-channel group, the value of i ranges from 0 to C / 4-1. Encoding mapping is performed using sine and cosine periodic functions of different frequencies. The encoding value at index=2i for even-numbered channels is sin(Δx / (10000^(2i / (C / 2)))), and the encoding value at index=2i+1 for odd-numbered channels is sin(Δx / (10000^(2i / (C / 2)))). The encoded value is cos(Δx / (10000^(2i / (C / 2)))); Similarly, the same form of sine / cosine periodic mapping is performed on the y-axis coordinate Δy in its corresponding sub-channel group to calculate the position encoding component in the y-axis direction; the position encoding components calculated along the x-axis and y-axis are concatenated along the channel dimension to form a two-dimensional sine / cosine position encoding matrix with dimension 1×C; based on the real-time two-dimensional spatial relative coordinate vector (Δx, Δy) and the two-dimensional sine / cosine position encoding matrix, the spatial position matrix of other traffic participants is determined.

[0047] S304. Based on the normalized networked sensing collaboration matrix and the spatial location matrix of other traffic participants, the spatial location query matrix of other traffic participants is obtained.

[0048] In this embodiment, the specific process of obtaining the spatial location query matrix of other traffic participants based on the normalized network-connected sensing cooperation matrix and the spatial location matrix of other traffic participants is as follows: The spatial location matrix processing module performs sine and cosine transformations on the heading angle in the normalized network-connected sensing cooperation matrix to obtain a heading angle unit vector; the position coordinates, velocity, and heading angle unit vector of the normalized network-connected sensing cooperation matrix are fused to obtain a highly sensitive implicit feature matrix; the acceleration data in the normalized network-connected sensing cooperation matrix is ​​processed to obtain a low-sensitivity acceleration matrix; the highly sensitive implicit feature matrix and the low-sensitivity acceleration matrix are fused to obtain the spatial location implicit feature matrix of other traffic participants; the spatial location implicit feature matrix of other traffic participants is fused with the spatial location matrix of other traffic participants to obtain the spatial location query matrix of other traffic participants.

[0049] For example, the spatial position matrix processing module splits the state variables in the normalized network sensing cooperative matrix into two parallel processing paths based on physical characteristics and noise sensitivity. For the position coordinates (x, y), velocity (v_x, v_y), and heading angle θ, to address the periodic jumps in the heading angle, a sine and cosine transformation is performed on the heading angle θ to obtain a heading angle unit vector (sinθ, cosθ). The position coordinates, velocity, and heading angle unit vectors are concatenated to construct a highly sensitive state vector V_state=[x, y, v_x, v_y, sinθ, cosθ]. The spatial position matrix processing module contains four fully connected linear layers, where the first and second fully connected linear layers increase the dimensionality of the highly sensitive state vector to 128 dimensions, resulting in a highly sensitive implicit feature matrix. For the acceleration state value a with high-frequency noise and jitter characteristics, a low-sensitivity state vector V_acc=[a_x, a_y] is constructed; the third fully connected linear layer increases the dimension of the low-sensitivity state vector V_acc to 32 dimensions to obtain the acceleration low-sensitivity matrix, so as to guide the network to focus on the macroscopic acceleration and deceleration trend and filter high-frequency noise.

[0050] The highly sensitive implicit feature matrix and the low-sensitivity acceleration matrix are concatenated along the channel dimension to generate a hybrid implicit feature vector of dimension 1×160. The fourth fully connected linear layer increases the dimension of the hybrid implicit feature vector from 160 to 256, completing the noise reduction and feature alignment of heterogeneous features, and obtaining the spatial location implicit feature matrix of other traffic participants. The spatial location implicit feature matrix of other traffic participants is then fused with the spatial location matrix of other traffic participants to obtain the spatial location query matrix of other traffic participants.

[0051] like Figure 4 As shown, the specific steps in S104, "determine the fusion representation feature key matrix of other traffic participants based on the key matrix of other traffic participants in the image, the road topology key matrix, and the spatial location query matrix of other traffic participants," are as follows: S401, using the spatial location query matrix of other traffic participants as the query basis, the image other traffic participant key matrix as the key, and the image other traffic participant value matrix as the value, conduct an interactive query to obtain the implicit query matrix of spatial location of other traffic participants.

[0052] In this embodiment, the pre-trained vehicle trajectory planning output model specifically includes an implicit query matrix processing module. Through the cross-attention head of this module, interactive queries are performed using the spatial location query matrix of other traffic participants as the query basis, the image key matrix of other traffic participants as the key, and the image value matrix of other traffic participants as the value, to obtain the implicit query matrix of the spatial location of other traffic participants. This allows for the association of network-connected sensing collaborative data with the image-sensed data of other traffic participants.

[0053] S402 uses the implicit query matrix of spatial location of other traffic participants as the query basis, the road topology key matrix as the key, and the road topology value matrix as the value to carry out interactive query and obtain the fusion representation feature matrix of other traffic participants.

[0054] In this embodiment, the pre-trained vehicle trajectory planning output model specifically includes a fusion representation feature matrix processing module. Through the cross-attention head of this module, interactive queries are performed using the implicit query matrix of other traffic participants' spatial locations as the query basis, the road topology key matrix as the key, and the road topology value matrix as the value, to obtain the fusion representation feature matrix of other traffic participants. This associates other traffic participants with the road topology, forming constraints between them.

[0055] S403, project the fusion representation feature matrix of other traffic participants to obtain the fusion representation feature key value matrix of other traffic participants.

[0056] In this embodiment, the specific process of projecting the fusion representation feature matrix of other traffic participants to obtain the fusion representation feature key-value matrix of other traffic participants is as follows: The fusion representation feature matrix processing module includes a fusion representation key projection layer and a fusion representation value projection layer; the fusion representation key matrix of other traffic participants is obtained by projecting the fusion representation feature matrix of other traffic participants onto the fusion representation feature matrix of other traffic participants through the fusion representation key projection layer, and the fusion representation value matrix of other traffic participants is obtained by projecting the fusion representation feature matrix of other traffic participants onto the fusion representation feature matrix of other traffic participants through the fusion representation value projection layer; the fusion representation key matrix of other traffic participants and the fusion representation value matrix of other traffic participants together constitute the fusion representation feature key-value matrix of other traffic participants. This process realizes the unified feature representation of other traffic participants within the field of view, the visual blind spot, and the beyond-line-of-sight range, providing a complete and continuous data foundation for the generation of autonomous vehicle planning driving trajectories.

[0057] like Figure 5 As shown, the specific steps in S105, "determine the vehicle's planned driving trajectory based on the fusion representation feature key-value matrix of other traffic participants, the image key-value matrix of other traffic participants, the road topology key-value matrix, and the preset vehicle planned driving trajectory query vector," are as follows: S501, using the preset autonomous vehicle planning trajectory query vector as the query basis, the fusion representation feature key matrix of other traffic participants as the key, and the fusion representation feature value matrix of other traffic participants as the value, perform interactive query to obtain the first autonomous vehicle planning trajectory decision feature matrix.

[0058] In this embodiment, the pre-trained vehicle trajectory planning output model specifically includes a self-planned driving trajectory query vector and a first decision feature matrix processing module; wherein, the self-planned driving trajectory query vector is a learnable vector. Through the cross-attention head of the first decision feature matrix processing module, using the preset self-planned driving trajectory query vector as the query basis, the fused representation feature key matrix of other traffic participants as the key, and the fused representation feature value matrix of other traffic participants as the value, an interactive query is performed to obtain the first self-planned driving trajectory decision feature matrix. Based on the fused representation feature key value matrix of other traffic participants within the visual field, blind spots, and beyond the line-of-sight range, the interactive query ensures that the first self-planned driving trajectory decision feature matrix fully includes other traffic participants in the visual blind spots due to physical occlusion or beyond the line-of-sight range, avoiding trajectory conflicts caused by missed detection of traffic participants due to image visual blind spots.

[0059] S502, using the first autonomous vehicle planning trajectory decision feature matrix as the query basis, the image other traffic participant key matrix as the key, and the image other traffic participant value matrix as the value, perform interactive query to obtain the second autonomous vehicle planning trajectory decision feature matrix.

[0060] In this embodiment, the pre-trained vehicle trajectory planning output model specifically includes a second decision feature matrix processing module. Through the cross-attention head of this module, an interactive query is performed using the first vehicle trajectory planning decision feature matrix as the query basis, the image other traffic participant key matrix as the key, and the image other traffic participant value matrix as the value, to obtain the second vehicle trajectory planning decision feature matrix. The first vehicle trajectory planning decision feature matrix is ​​then interactively queried again with the image other traffic participant key-value matrix obtained from surrounding image data. This ensures that the second vehicle trajectory planning decision feature matrix simultaneously includes features of other traffic participants from both connected perception collaborative data and surrounding image data, improving the accuracy of the vehicle trajectory planning response to other traffic participants visible within the vehicle's camera's line of sight.

[0061] S503, using the second autonomous vehicle planning trajectory decision feature matrix as the query basis, the road topology key-value matrix as the key, and the road topology value matrix as the value, perform an interactive query to obtain the third autonomous vehicle planning trajectory decision feature matrix.

[0062] In this embodiment, the pre-trained vehicle trajectory planning output model specifically includes a third decision feature matrix processing module. Through the cross-attention head of this module, an interactive query is performed using the second vehicle trajectory planning decision feature matrix as the query basis, the road topology key-value matrix as the key, and the road topology value matrix as the value, to obtain the third vehicle trajectory planning decision feature matrix. The interactive query between the second and road topology key-value matrices binds the third vehicle trajectory planning decision feature matrix to road topology features, providing road topology constraints for the vehicle trajectory planning and reducing the generation of vehicle trajectories that deviate from the road topology.

[0063] S504, based on the first vehicle planning driving trajectory decision feature matrix, the second vehicle planning driving trajectory decision feature matrix, and the third vehicle planning driving trajectory decision feature matrix, determine the vehicle's planned driving trajectory.

[0064] In this embodiment, the process of determining the vehicle's planned driving trajectory based on the first, second, and third vehicle planned driving trajectory decision feature matrices is as follows: A pre-trained vehicle planned driving trajectory output model includes a decision fusion module; the first, second, and third vehicle planned driving trajectory decision feature matrices are fused through the decision fusion module to obtain a fused vehicle planned driving trajectory decision feature matrix; the fused vehicle planned driving trajectory decision feature matrix is ​​then dimensionality-reduced to obtain a set of relative position coordinates for the vehicle in future time periods; and the set of relative position coordinates for the vehicle in future time periods is then integrated to obtain the vehicle's planned driving trajectory. By fusing the decision feature matrices of the three-way vehicle planning trajectory through the decision fusion module, and integrating the environmental data from the key-value matrices of other traffic participants, the key-value matrices of other traffic participants in the image, and the key-value matrices of the road topology, the vehicle's relative position coordinates for future time periods are output after dimensionality reduction processing. This ensures that the vehicle's planned driving trajectory includes data from other traffic participants visible within the field of view as well as those in the blind spot, thereby improving vehicle driving safety in occluded and beyond-line-of-sight scenarios.

[0065] This embodiment provides a method for generating a vehicle's planned driving trajectory. The specific execution process of this method is as follows: Equipped in intelligent connected vehicles, the vehicle is equipped with onboard cameras, C V2X communication components and onboard computing unit; a pre-trained vehicle trajectory planning output model. The vehicle trajectory planning output model sequentially includes an image processing module, a normalization processing module, a spatial position matrix processing module, an implicit query matrix processing module, a fusion representation feature matrix processing module, a first decision feature matrix processing module, a second decision feature matrix processing module, a third decision feature matrix processing module, and a decision fusion module. The vehicle trajectory planning output model has a built-in learnable parameter, the vehicle's planned driving trajectory query vector.

[0066] The image processing module includes a ResNet-50 backbone network, a Feature Pyramid Network (FPN), and a BEVFormer network, connected in series with traffic participant projection layers and road topology projection layers. The traffic participant projection layer contains a key projection layer and a value projection layer, while the road topology projection layer contains a key projection layer and a value projection layer. The image processing module takes surrounding image data as input and outputs a key matrix of other traffic participants, a value matrix of other traffic participants, a key matrix of road topology, and a value matrix of road topology. The normalization processing module takes connected vehicle sensing collaborative data as input and outputs a normalized connected vehicle sensing collaborative matrix. The spatial location matrix processing module has four fully connected linear layers that sequentially perform relative coordinate calculation, sine and cosine position encoding, high and low sensitivity feature splitting, and feature concatenation and dimensionality enhancement. The spatial location matrix processing module takes vehicle status data and a normalized connected vehicle sensing collaborative matrix as input and outputs a spatial location query matrix for other traffic participants.

[0067] The implicit query matrix processing module is configured with a cross-attention head. It takes into account the spatial location query matrix of other traffic participants, the image key matrix of other traffic participants, and the image value matrix of other traffic participants, and outputs the implicit query matrix of spatial location of other traffic participants. The fusion representation feature matrix processing module is configured with a cross-attention head, and its backend is connected in series with a fusion representation key projection layer and a fusion representation value projection layer. It takes into account the implicit query matrix of spatial location of other traffic participants, the road topology key matrix, and the road topology value matrix, and outputs the fusion representation feature key matrix and the fusion representation feature value matrix of other traffic participants. The first decision feature matrix processing module is configured with a cross-attention head. It takes into account the vehicle's planned driving trajectory query vector, the fusion representation feature key matrix of other traffic participants, and the fusion representation feature value matrix of other traffic participants, and outputs the first decision feature matrix of the vehicle's planned driving trajectory.

[0068] The second decision feature matrix processing module is configured with a cross-attention head. It takes as input the first vehicle's planned driving trajectory decision feature matrix, the image other traffic participant key matrix, and the image other traffic participant value matrix, and outputs the second vehicle's planned driving trajectory decision feature matrix. The third decision feature matrix processing module is also configured with a cross-attention head. It takes as input the second vehicle's planned driving trajectory decision feature matrix, the road topology key matrix, and the road topology value matrix, and outputs the third vehicle's planned driving trajectory decision feature matrix. The decision fusion module takes the first, second, and third vehicle's planned driving trajectory decision feature matrices as input, outputs the set of relative position coordinates of the vehicle over future time periods, and integrates them to obtain the vehicle's planned driving trajectory.

[0069] The data flow connection is as follows: Surrounding image data is input into the image processing module to obtain the image other traffic participant key-value matrix and the road topology key-value matrix; Connected sensing collaborative data is input into the normalization processing module, and vehicle status data and the normalized connected sensing collaborative matrix are jointly input into the spatial location matrix processing module to obtain the other traffic participant spatial location query matrix. The other traffic participant spatial location query matrix and the image other traffic participant key-value matrix are input into the implicit query matrix processing module to obtain the other traffic participant spatial location implicit query matrix; The other traffic participant spatial location implicit query matrix and the road topology key-value matrix are input into the fusion representation feature matrix processing module to obtain the other traffic participant fusion representation feature key-value matrix.

[0070] The vehicle's planned driving trajectory query vector and the key-value matrix of other traffic participants' fused representation features are input into the first decision feature matrix processing module to obtain the first vehicle planned driving trajectory decision feature matrix. The first vehicle planned driving trajectory decision feature matrix and the key-value matrix of other traffic participants in the image are input into the second decision feature matrix processing module to obtain the second vehicle planned driving trajectory decision feature matrix. The second vehicle planned driving trajectory decision feature matrix and the road topology key-value matrix are input into the third decision feature matrix processing module to obtain the third vehicle planned driving trajectory decision feature matrix. The three decision feature matrices are jointly sent to the decision fusion module to output the vehicle's planned driving trajectory. Based on the above vehicle planned driving trajectory output model architecture, it can simultaneously integrate image perception information and network perception collaborative data, so that the input for generating the vehicle planned driving trajectory includes data of other traffic participants who are in the visual blind spot due to physical occlusion or beyond line of sight, avoiding the collision risk caused by simply relying on surrounding image data to generate the vehicle planned driving trajectory.

[0071] The training process for the vehicle trajectory output model is as follows: A training dataset is constructed, collecting a large number of real traffic scene samples. These samples include surrounding image data, vehicle state data, network perception collaborative data from other traffic participants, and labeled ground truth trajectories. Scenes cover intersection occlusion, vehicle blind spots, long-distance beyond-line-of-sight encounters, and ordinary open roads. The network perception collaborative data is generated by other vehicles (C). V2X device broadcasts are obtained through Protobuf serialization transmission and deserialization parsing.

[0072] Initialize the parameters of the vehicle trajectory planning output model, including the weights of the ResNet-50 backbone network, FPN, and BEVFormer network. Also initialize the parameters of each projection layer, the four fully connected linear layers, and all cross-attention heads. Randomly initialize the vehicle's planned driving trajectory query vector, which is then continuously optimized as a learnable parameter during training iterations. Perform forward propagation, inputting the surrounding image data, vehicle state data, and network-connected perception collaborative data of a single training sample into the vehicle trajectory planning output model. The model then outputs the predicted vehicle's planned driving trajectory.

[0073] The loss function is calculated and backpropagated to construct a trajectory regression loss function. The error between the predicted vehicle planned trajectory and the labeled ground truth trajectory is calculated. The gradient descent algorithm is used to backpropagate and update all parameters of the vehicle planned trajectory output model. The vehicle planned trajectory query vector is simultaneously iteratively optimized. Batch samples are fed in for iterative training. When the validation set loss no longer decreases after several rounds, training is terminated, and the pre-trained vehicle planned trajectory output model is obtained. All network weights and the trained vehicle planned trajectory query vector are saved.

[0074] The real-vehicle reasoning scenario is an urban intersection. As the vehicle approaches the intersection, a large truck physically obstructs the view from the left. Behind the truck is a non-motorized vehicle, which is in the blind spot of the onboard camera and cannot be recognized by the image. The non-motorized vehicle is equipped with a C... V2X communication components. The vehicle's onboard camera collects surrounding image data, and the vehicle's bus reads the vehicle's position coordinates to obtain vehicle status data. The vehicle's C... The V2X communication component receives serialized V2X data broadcast by non-motorized vehicles, deserializes it using Protobuf, and obtains the non-motorized vehicle's position coordinates, speed, acceleration, and heading angle to obtain connected sensing and collaborative data.

[0075] Surrounding image data is fed into the image processing module, where feature extraction and BEV space transformation are performed via ResNet-50, FPN, and BEVFormer to obtain the other traffic participants matrix and road topology matrix. These are then projected through a traffic participant projection layer and a road topology projection layer to generate the image other traffic participant key matrix, image other traffic participant value matrix, road topology key matrix, and road topology value matrix. At this point, image perception can only identify trucks ahead and cannot identify non-motorized vehicles in the blind spot behind the trucks. The normalization processing module performs dual-scale normalization on the network perception collaborative data to achieve scale alignment and retain non-motorized vehicle data in the blind spot that exceeds the visual scale. The spatial position matrix processing module uses the vehicle and non-motorized vehicle position coordinates to solve for the relative position vector (Δx, Δy), and uses sine and cosine functions to complete position encoding to obtain the spatial position matrix. High and low sensitivity feature separation processing is performed on the network state variables, and the resulting matrix is ​​fused to obtain the other traffic participant spatial position query matrix. This matrix contains complete motion information of non-motorized vehicles in the blind spot.

[0076] The implicit query matrix processing module utilizes cross-attention heads, using the spatial location query matrix of other traffic participants as the query basis, and interactively queries with the key-value matrix of other traffic participants in the image to obtain the implicit query matrix of the spatial location of other traffic participants, realizing the mutual correlation between network-connected perception collaborative data and other traffic participants in image perception. The fusion representation feature matrix processing module again uses cross-attention heads combined with the road topology key-value matrix to obtain the fusion representation feature matrix of other traffic participants, establishing the constraint relationship between other traffic participants and the road topology structure. Through the fusion representation key projection layer and the fusion representation value projection layer, the fusion representation feature key-value matrix of other traffic participants is generated. The fusion representation feature key-value matrix of other traffic participants simultaneously contains information on trucks, non-motorized vehicles in the blind spot, and road topology within the field of view, realizing the unified feature representation of other traffic participants within the field of view, the visual blind spot, and beyond the line of sight.

[0077] The first decision feature matrix processing module uses the trained vehicle's planned driving trajectory query vector as the query basis, and interacts with other traffic participants' fusion representation feature key-value matrices to obtain the first vehicle's planned driving trajectory decision feature matrix. This fully incorporates information from other traffic participants in the visual blind spot, preventing trajectory conflicts caused by missed detection of traffic participants due to image visual blind spots from the source. The second decision feature matrix processing module combines the key-value matrices of other traffic participants in the image for a second interactive query to obtain the second vehicle's planned driving trajectory decision feature matrix, improving the response accuracy of the vehicle's planned driving trajectory to visible targets within the vehicle's camera's line of sight. The third decision feature matrix processing module combines the road topology key-value matrix to complete a third interactive query to obtain the third vehicle's planned driving trajectory decision feature matrix. This imposes road topology constraints on the vehicle's planned driving trajectory, reducing the generation of vehicle planned driving trajectories that deviate from the road topology.

[0078] The three-way decision feature matrix is ​​input into the decision fusion module. After fusion and dimensionality reduction, it outputs a set of continuous relative position coordinates of the vehicle within a specified future time period, which is integrated to form the vehicle's planned driving trajectory. The final output vehicle planned driving trajectory actively reserves a safe distance from non-motorized vehicles in blind spots to avoid the risk of collisions in blind spots. The vehicle planned driving trajectory is constrained within the lane range and will not generate a vehicle planned driving trajectory that deviates from the road topology, effectively improving the safety and rationality of trajectory planning in occluded and beyond-line-of-sight scenarios.

[0079] In another embodiment of this application, a vehicle planning and driving trajectory generation device is provided, the device as follows: Figure 6 As shown, it includes: a data acquisition module 601, a first key-value matrix determination module 602, a spatial location query matrix determination module 603, a second key-value matrix determination module 604, and a vehicle planning and driving trajectory generation module 605.

[0080] The data acquisition module 601 is used to acquire surrounding image data, vehicle status data, and network-connected perception and collaborative data of other traffic participants. The first key value matrix determination module 602 is used to determine the key value matrix of other traffic participants and the road topology key value matrix based on the surrounding image data. The spatial location query matrix determination module 603 is used to determine the spatial location query matrix of other traffic participants based on the vehicle status data and the network-connected sensing collaborative data. The second key value matrix determination module 604 is used to determine the fusion representation feature key value matrix of other traffic participants based on the key value matrix of other traffic participants in the image, the road topology key value matrix, and the spatial location query matrix of other traffic participants. The vehicle planning and driving trajectory generation module 605 is used to determine the vehicle planning and driving trajectory based on the fusion representation feature key value matrix of other traffic participants, the image other traffic participant key value matrix, the road topology key value matrix, and the preset vehicle planning and driving trajectory query vector.

[0081] In another embodiment of this application, a vehicle is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. Memory, used to store computer programs; When the processor executes the program stored in the memory, it implements the vehicle planning and driving trajectory generation method described in any of the foregoing method embodiments.

[0082] The vehicle provided in this application embodiment has a processor that, by executing a program stored in its memory, acquires surrounding image data, vehicle status data, and network-connected perception and coordination data of other traffic participants; based on the surrounding image data, it determines a key-value matrix of other traffic participants and a road topology key-value matrix; based on the vehicle status data and the network-connected perception and coordination data, it determines a spatial location query matrix of other traffic participants; based on the key-value matrix of other traffic participants, the road topology key-value matrix, and the spatial location query matrix of other traffic participants, it determines a fusion representation feature key-value matrix of other traffic participants; based on the fusion representation feature key-value matrix of other traffic participants, the key-value matrix of other traffic participants, the road topology key-value matrix, and a preset self-planned driving trajectory query vector, it determines the self-planned driving trajectory of the vehicle; and by acquiring the network-connected perception and coordination data of other traffic participants... By perceiving and coordinating data, a spatial location query matrix of other traffic participants is generated, which can represent the spatial location of the target in the blind spot. This other traffic participant spatial location query matrix is ​​then fused with the image other traffic participant key-value matrix obtained from the vehicle's surrounding image data, as well as the road topology key-value matrix, to form a fused representation feature key-value matrix of other traffic participants. This ensures that the input for generating the driving trajectory includes not only traffic participants visible within the vehicle's camera's line of sight, but also data of other traffic participants in the visual blind spot due to physical occlusion or beyond the line of sight. Based on the fused representation feature key-value matrix of other traffic participants, and combined with a preset vehicle planning driving trajectory query vector, the driving trajectory is generated. This avoids trajectory conflicts caused by missed detection of traffic participants due to image visual blind spots, thus solving the collision risk problem of simply relying on environmental images to generate trajectories and improving vehicle driving safety in occluded and beyond-line-of-sight scenarios.

[0083] The communication bus 704 mentioned in the above-mentioned vehicle can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 704 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0084] Communication interface 702 is used for communication between the aforementioned vehicle and other devices.

[0085] The memory 703 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0086] The processor 701 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0087] In another embodiment of this application, a computer-readable storage medium is provided, on which a program for a vehicle planning and driving trajectory generation method is stored. When the program for the vehicle planning and driving trajectory generation method is executed by a processor, it implements the steps of the vehicle planning and driving trajectory generation method described in any of the foregoing method embodiments.

[0088] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for generating a vehicle's planned driving trajectory, characterized in that, include: Acquire surrounding image data, vehicle status data, and networked perception and collaborative data from other traffic participants; Based on the surrounding image data, determine the key-value matrix of other traffic participants in the image and the key-value matrix of the road topology; Based on the vehicle status data and the network-connected sensing collaborative data, a spatial location query matrix for other traffic participants is determined. Based on the image other traffic participant key value matrix, the road topology key value matrix, and the other traffic participant spatial location query matrix, determine the other traffic participant fusion representation feature key value matrix; The vehicle's planned driving trajectory is determined based on the fusion representation feature key matrix of other traffic participants, the image key matrix of other traffic participants, the road topology key matrix, and the preset vehicle planned driving trajectory query vector.

2. The method according to claim 1, characterized in that, The step of determining the key-value matrix of other traffic participants and the road topology key-value matrix based on the surrounding image data includes: Other traffic participant features and road topology features are extracted from the surrounding image data to obtain other traffic participant matrix and road topology matrix; The other traffic participant matrix and the road topology matrix are projected to obtain the image other traffic participant key matrix and the road topology key matrix, respectively.

3. The method according to claim 1, characterized in that, The step of determining the spatial location query matrix of other traffic participants based on the vehicle status data and the network-connected sensing collaborative data includes: The network-connected sensing collaborative data is subjected to dual-scale normalization processing to obtain a normalized network-connected sensing collaborative matrix; Based on the position coordinates in the vehicle status data and the position coordinates in the connected sensing and coordination data, the relative position coordinates of other traffic participants relative to the vehicle are determined. Based on the relative position coordinates of the other traffic participants with respect to the vehicle, determine the spatial position matrix of the other traffic participants; Based on the normalized networked sensing collaboration matrix and the spatial location matrix of other traffic participants, the spatial location query matrix of other traffic participants is obtained.

4. The method according to claim 3, characterized in that, The process of performing dual-scale normalization on the network-connected sensing collaborative data to obtain a normalized network-connected sensing collaborative matrix includes: Determine the collaborative scale corresponding to the network-connected sensing collaborative data, and the visual scale corresponding to the surrounding image data; The data mapping ratio is determined based on the collaborative scale and the visual scale; The network-connected sensing collaborative data is scale-mapped according to the mapping ratio to obtain a normalized network-connected sensing collaborative matrix.

5. The method according to claim 4, characterized in that, The step of obtaining the spatial location query matrix of other traffic participants based on the normalized networked sensing cooperation matrix and the spatial location matrix of other traffic participants includes: The heading angle in the normalized network sensing cooperative matrix is ​​transformed by sine and cosine to obtain a unit vector of heading angle. By fusing the position coordinates, velocity, and heading angle unit vector of the normalized network sensing cooperative matrix, a highly sensitive implicit feature matrix is ​​obtained; The acceleration data in the normalized network sensing cooperative matrix is ​​processed to obtain a low-sensitivity acceleration matrix; By fusing the highly sensitive implicit feature matrix with the low-sensitivity acceleration matrix, the spatial location implicit feature matrix of other traffic participants is obtained; By fusing the implicit feature matrix of the spatial location of the other traffic participants with the spatial location matrix of the other traffic participants, a spatial location query matrix of the other traffic participants is obtained.

6. The method according to claim 1, characterized in that, The step of determining the fusion representation feature key matrix of other traffic participants based on the image other traffic participant key matrix, the road topology key matrix, and the spatial location query matrix of other traffic participants includes: Using the spatial location query matrix of other traffic participants as the query basis, the image other traffic participant key matrix as the key, and the image other traffic participant value matrix as the value, an interactive query is performed to obtain the implicit query matrix of spatial location of other traffic participants; Using the implicit query matrix of spatial location of other traffic participants as the query basis, the road topology key matrix as the key, and the road topology value matrix as the value, an interactive query is carried out to obtain the fusion representation feature matrix of other traffic participants; The fusion representation feature matrix of the other traffic participants is projected to obtain the fusion representation feature key value matrix of the other traffic participants.

7. The method according to claim 1, characterized in that, The step of determining the vehicle's planned driving trajectory based on the fused representation feature key-value matrix of other traffic participants, the image key-value matrix of other traffic participants, the road topology key-value matrix, and the preset vehicle planned driving trajectory query vector includes: Using the preset autonomous vehicle planning trajectory query vector as the query basis, the fusion representation feature key matrix of other traffic participants as the key, and the fusion representation feature value matrix of other traffic participants as the value, an interactive query is performed to obtain the first autonomous vehicle planning trajectory decision feature matrix. Using the first autonomous vehicle planning and driving trajectory decision feature matrix as the query basis, the image other traffic participant key matrix as the key, and the image other traffic participant value matrix as the value, an interactive query is performed to obtain the second autonomous vehicle planning and driving trajectory decision feature matrix. Using the second autonomous vehicle planning and driving trajectory decision feature matrix as the query basis, the road topology key-value matrix as the key, and the road topology value matrix as the value, an interactive query is performed to obtain the third autonomous vehicle planning and driving trajectory decision feature matrix; The vehicle's planned driving trajectory is determined based on the first vehicle planning driving trajectory decision feature matrix, the second vehicle planning driving trajectory decision feature matrix, and the third vehicle planning driving trajectory decision feature matrix.

8. The method according to claim 7, characterized in that, The step of determining the vehicle's planned driving trajectory based on the first vehicle planning driving trajectory decision feature matrix, the second vehicle planning driving trajectory decision feature matrix, and the third vehicle planning driving trajectory decision feature matrix includes: By fusing the first autonomous vehicle planning driving trajectory decision feature matrix, the second autonomous vehicle planning driving trajectory decision feature matrix, and the third autonomous vehicle planning driving trajectory decision feature matrix, a vehicle fusion planning driving trajectory decision feature matrix is ​​obtained. The dimensionality reduction process is performed on the vehicle fusion planning driving trajectory decision feature matrix to obtain the set of relative position coordinates of the vehicle in future time periods; The vehicle's planned driving trajectory is obtained by integrating the set of relative position coordinates of the vehicle in the future time period.

9. A vehicle planning and driving trajectory generation device, characterized in that, include: The data acquisition module is used to acquire surrounding image data, vehicle status data, and networked perception and collaborative data from other traffic participants. The first key value matrix determination module is used to determine the key value matrix of other traffic participants and the road topology key value matrix based on the surrounding image data. The spatial location query matrix determination module is used to determine the spatial location query matrix of other traffic participants based on the vehicle status data and the network-connected sensing collaborative data. The second key value matrix determination module is used to determine the fusion representation feature key value matrix of other traffic participants based on the key value matrix of other traffic participants in the image, the road topology key value matrix, and the spatial location query matrix of other traffic participants; The vehicle planning and driving trajectory generation module is used to determine the vehicle's planned driving trajectory based on the fusion representation feature key value matrix of other traffic participants, the image other traffic participant key value matrix, the road topology key value matrix, and the preset vehicle planning and driving trajectory query vector.

10. A vehicle, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor, when executing a program stored in memory, implements the vehicle planning and driving trajectory generation method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for a vehicle planning and driving trajectory generation method, which, when executed by a processor, implements the steps of the vehicle planning and driving trajectory generation method according to any one of claims 1-8.

Images