Training method of scene coding model, trajectory planning method and device

Abstract

The present disclosure provides a scene encoding model training method and device, and a trajectory planning method and device, relates to the technical field of artificial intelligence, and in particular to the technical field of autonomous driving. The implementation scheme is: obtaining a first scene sample pair, the first scene sample pair including first information of a first driving scene and second information of a second driving scene; inputting the first information into a scene encoding model to obtain a first scene encoding output by the scene encoding model; inputting the second information into the scene encoding model to obtain a second scene encoding output by the scene encoding model; determining a loss value of the scene encoding model based on a first similarity between the first scene encoding and the second scene encoding; and adjusting parameters of the scene encoding model based on the loss value.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of autonomous driving technology, and specifically to a training method and device for a scene coding model, a trajectory planning method and device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Autonomous driving technology involves multiple aspects, including environmental perception, behavioral decision-making, trajectory planning, and motion control. Relying on the collaborative efforts of sensors, visual computing systems, and positioning systems, autonomous vehicles can operate autonomously with minimal or no driver input. Trajectory planning is a hot topic within the autonomous driving field.

[0003] The approaches described in this section are not necessarily approaches that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any approach described in this section is prior art simply by virtue of its inclusion in this section. Similarly, unless otherwise indicated, the issues raised in this section should not be considered as having been recognized in any prior art. Summary of the Invention

[0004] The present disclosure provides a scene coding model training method and device, a trajectory planning method and device, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of the present disclosure, a method for training a scene coding model is provided, comprising: obtaining a first scene sample pair, wherein the first scene sample pair includes first information of a first driving scene and second information of a second driving scene, the number of obstacles in the second driving scene that are different from the first driving scene is less than a first threshold, and a distance between a second future trajectory of a vehicle in the second driving scene and the first future trajectory of the vehicle in the first driving scene is greater than a second threshold; inputting the first information into the scene coding model to obtain a first scene coding output by the scene coding model; inputting the second information into the scene coding model to obtain a second scene coding output by the scene coding model; determining a loss value of the scene coding model based on a first similarity between the first scene coding and the second scene coding; and adjusting parameters of the scene coding model based on the loss value.

[0006] According to one aspect of the present disclosure, there is provided a method for obtaining current driving scene information of a vehicle; inputting the driving scene information into a trained scene coding model to generate a current scene coding, wherein the scene coding model is trained based on a training method for a scene coding model of an embodiment of the present disclosure; respectively calculating similarities between the current scene coding and a plurality of historical scene codings; and determining a planned trajectory of the vehicle in the current driving scene based on a future trajectory corresponding to a target historical scene coding, wherein the target historical scene coding is a historical scene coding having a similarity with the current scene coding greater than a threshold.

[0007] According to one aspect of the present disclosure, a training device for a scene coding model is provided, comprising: a first acquisition module configured to acquire a first scene sample pair, wherein the first scene sample pair includes first information of a first driving scene and second information of a second driving scene, the number of obstacles in the second driving scene that are different from the first driving scene is less than a first threshold, and a distance between a second future trajectory of a vehicle in the second driving scene and the first future trajectory of the vehicle in the first driving scene is greater than a second threshold; a first input module configured to input the first information into the scene coding model to obtain a first scene coding output by the scene coding model; a second input module configured to input the second information into the scene coding model to obtain a second scene coding output by the scene coding model; a determination module configured to determine a loss value of the scene coding model based on a first similarity between the first scene coding and the second scene coding; and an adjustment module configured to adjust parameters of the scene coding model based on the loss value.

[0008] According to one aspect of the present disclosure, a trajectory planning device is provided, comprising: an acquisition module configured to acquire current driving scene information of a vehicle; a generation module configured to input the driving scene information into a trained scene coding model to generate a current scene coding, wherein the scene coding model is trained by a training device of a scene coding model based on an embodiment of the present disclosure; a calculation module configured to respectively calculate the similarity between the current scene coding and a plurality of historical scene codings; and a determination module configured to determine the planned trajectory of the vehicle in the current driving scene based on a future trajectory corresponding to a target historical scene coding, wherein the target historical scene coding is a historical scene coding whose similarity to the current scene coding is greater than a threshold.

[0009] According to one aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any of the methods described above.

[0010] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute any of the above methods.

[0011] According to one aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions implement any of the above methods when executed by a processor.

[0012] According to one or more embodiments of the present disclosure, the accuracy of the scene coding model and the efficiency and accuracy of trajectory planning can be improved.

[0013] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings illustrate exemplary embodiments and constitute a part of the specification. Together with the description of the specification, they serve to explain exemplary implementation of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals designate similar, but not necessarily identical, elements.

[0015] Figure 1 A schematic diagram illustrating an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0016] Figure 2 A flowchart of a method for training a scene coding model according to an embodiment of the present disclosure is shown;

[0017] Figure 3 A schematic diagram illustrating an obstacle modification method according to an embodiment of the present disclosure is shown;

[0018] Figure 4 A schematic diagram showing the trajectory change of an obstacle after modification according to an embodiment of the present disclosure is shown;

[0019] Figure 5 A flow chart of a trajectory planning method according to an embodiment of the present disclosure is shown;

[0020] Figure 6A structural block diagram of a training device for a scene coding model according to an embodiment of the present disclosure is shown;

[0021] Figure 7 shows a structural block diagram of a trajectory planning device according to an embodiment of the present disclosure; and

[0022] Figure 8 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0023] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] In this disclosure, unless otherwise specified, the use of terms such as "first" and "second" to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements. Such terms are only used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, while in some cases, based on the context of the description, they may also refer to different instances.

[0025] The terms used in the descriptions of the various examples described in this disclosure are for the purpose of describing specific examples only and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element may be one or more. In addition, the term "and / or" used in this disclosure encompasses any one and all possible combinations of the listed items.

[0026] In related technologies, a scene coding model can be used to represent scene information during vehicle driving (e.g., surrounding obstacle information, lane information, etc.) as scene codes. A database is then retrieved to identify historical scenes with a high degree of similarity to the current scene, and the driving trajectory corresponding to the historical scene is used as the current planned trajectory. The scene coding model represents similar scene information as similar scene codes.

[0027] Some scenarios (for example, Scene A and Scene B) appear superficially similar, differing only by a single obstacle. However, this single obstacle may be critical and significantly impact the vehicle's driving decisions, resulting in the vehicle planning distinct trajectories in the two scenarios. The scene encoding models used in related technologies generate similar scene encodings for similar Scenes A and B, but fail to accurately describe the differences in key obstacles within the scenes, making it impossible to distinguish between them and, consequently, unable to plan appropriate obstacle avoidance trajectories.

[0028] To address the above issues, the present disclosure provides a method for training a scene coding model, which can enhance the scene coding model's ability to perceive key obstacles and improve the accuracy of scene coding. Based on the trained scene coding model, the present disclosure also provides a trajectory planning method, which can utilize the scene coding model for trajectory planning, improving the efficiency and accuracy of trajectory planning.

[0029] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1 FIG2 is a schematic diagram of an exemplary system 100 in which the various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes a motor vehicle 110 , a server 120 , and one or more communication networks 130 coupling the motor vehicle 110 to the server 120 .

[0031] In an embodiment of the present disclosure, the motor vehicle 110 may include a computing device according to an embodiment of the present disclosure and / or be configured to perform a method according to an embodiment of the present disclosure.

[0032] The server 120 may run one or more services or software applications that enable execution of the scene coding model training method and / or trajectory planning method. In some embodiments, the server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. The user of the motor vehicle 110 may, in turn, utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0033] Server 120 may include one or more general-purpose computers, specialized server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0034] The computing units in the server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

[0035] In some embodiments, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from motor vehicle 110. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of motor vehicle 110.

[0036] The network 130 may be any type of network known to those skilled in the art that can support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 130 may be a satellite communication network, a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (including, for example, Bluetooth, Wi-Fi), and / or any combination of these and other networks.

[0037] The system 100 may also include one or more databases 150. In some embodiments, these databases can be used to store data and other information. For example, one or more of the databases 150 can be used to store information such as audio files and video files. The data repository 150 can reside in a variety of locations. For example, the data repository used by the server 120 can be local to the server 120, or can be remote from the server 120 and can communicate with the server 120 via a network-based or dedicated connection. The data repository 150 can be of different types. In some embodiments, the data repository used by the server 120 can be a database, such as a relational database. One or more of these databases can store, update, and retrieve data to and from the database in response to commands.

[0038] In some embodiments, one or more of the databases 150 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0039] Motor vehicle 110 may include sensors 111 for sensing its surroundings. Sensors 111 may include one or more of the following: visual cameras, infrared cameras, ultrasonic sensors, millimeter-wave radar, and laser radar (LiDAR). Different sensors offer different detection accuracy and range. Cameras may be mounted on the front, rear, or other locations of the vehicle. Visual cameras can capture real-time information about the vehicle's interior and exterior and present it to the driver and / or passengers. Furthermore, by analyzing the images captured by the visual cameras, information such as traffic light indications, intersection conditions, and the operating status of other vehicles can be obtained. Infrared cameras can detect objects in night vision conditions. Ultrasonic sensors can be mounted on all sides of the vehicle, utilizing the strong directionality of ultrasonic waves to measure the distance of external objects from the vehicle. Millimeter-wave radars can be mounted on the front, rear, or other locations of the vehicle, utilizing the properties of electromagnetic waves to measure the distance of external objects from the vehicle. LiDARs can be mounted on the front, rear, or other locations of the vehicle, detecting object edges and shapes for object recognition and tracking. Due to the Doppler effect, radar devices can also measure changes in the speed of the vehicle and moving objects.

[0040] The motor vehicle 110 may also include a communication device 112. The communication device 112 may include a satellite positioning module that can receive satellite positioning signals (e.g., Beidou, GPS, GLONASS, and GALILEO) from satellites 141 and generate coordinates based on these signals. The communication device 112 may also include a module for communicating with a mobile communication base station 142. The mobile communication network may implement any suitable communication technology, such as GSM / GPRS, CDMA, LTE, and other current or evolving wireless communication technologies (e.g., 5G technology). The communication device 112 may also have a vehicle-to-everything (V2X) module that is configured to implement vehicle-to-vehicle (V2V) communication with other vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144, for example. In addition, the communication device 112 may also include a module configured to communicate with a user terminal 145 (including but not limited to a smartphone, tablet computer, or wearable device such as a watch) via a wireless local area network or Bluetooth using the IEEE 802.11 standard, for example. Using the communication device 112, the motor vehicle 110 may also access the server 120 via the network 130.

[0041] The motor vehicle 110 may also include a control device 113. The control device 113 may include a processor that communicates with various types of computer-readable storage devices or media, such as a central processing unit (CPU) or a graphics processing unit (GPU), or other dedicated processors. The control device 113 may include an autonomous driving system for automatically controlling various actuators in the vehicle. The autonomous driving system is configured to control the powertrain, steering system, and braking system of the motor vehicle 110 (not shown) via multiple actuators in response to input from multiple sensors 111 or other input devices to control acceleration, steering, and braking, respectively, without human intervention or limited human intervention. Some processing functions of the control device 113 may be implemented through cloud computing. For example, some processing may be performed using an on-board processor, while other processing may be performed using computing resources in the cloud. The control device 113 may be configured to execute the method according to the present disclosure. In addition, the control device 113 may be implemented as an example of a computing device on the motor vehicle side (client) according to the present disclosure.

[0042] Figure 1 The system 100 may be configured and operated in various ways to enable application of the various methods and apparatuses described in accordance with the present disclosure.

[0043] According to some embodiments, the motor vehicle 110 or the server 120 may execute the trajectory planning method of the embodiments of the present disclosure to generate a planned trajectory for the motor vehicle 110 based on a trained scene coding model.

[0044] According to some embodiments, the scene coding model can be trained by the server 120 or by other servers ( Figure 1 In other words, the training method of the scene coding model of the embodiment of the present disclosure can be executed by the server 120, or by other servers or the motor vehicle 110.

[0045] It should be noted that the server that executes the trajectory planning method of the embodiment of the present disclosure and the server that executes the training method of the scene coding model of the embodiment of the present disclosure can be the same server (for example, server 120) or different servers (for example, the trajectory planning method is executed by server 120, and the training method of the scene coding model is executed by other servers different from server 120).

[0046] According to an embodiment of the present disclosure, a method for training a scene coding model is provided. Figure 2 FIG2 shows a flow chart of a method 200 for training a scene coding model according to an embodiment of the present disclosure. The execution body of each step of the method 200 is usually a server (e.g. Figure 1 120 shown in FIG.

[0047] like Figure 2 As shown, the method 200 includes steps S210 - S250 .

[0048] In step S210, a first scene sample pair is obtained, the first scene sample pair including first information of a first driving scene and second information of a second driving scene. The number of obstacles in the second driving scene that differ from the first driving scene is less than a first threshold, and a distance between a second future trajectory of the vehicle in the second driving scene and the first future trajectory of the vehicle in the first driving scene is greater than a second threshold.

[0049] In step S220, the first information is input into the scene coding model to obtain a first scene code output by the scene coding model.

[0050] In step S230, the second information is input into the scene coding model to obtain a second scene code output by the scene coding model.

[0051] In step S240 , a loss value of a scene coding model is determined based on a first similarity between the first scene coding and the second scene coding.

[0052] In step S250, parameters of the scene coding model are adjusted based on the loss value.

[0053] According to an embodiment of the present disclosure, the first driving scenario and the second driving scenario differ only in a small number of obstacles (less than a first threshold), but their future trajectories differ significantly (greater than a second threshold). Training a scene coding model based on first scene sample pairs consisting of first and second information can improve the scene coding model's ability to perceive key obstacles in the scene, improve the accuracy of the scene coding, and thus improve the accuracy of the planned trajectory generated based on the scene coding.

[0054] The following describes each step of method 200 in detail.

[0055] In step S210, a first scene sample pair is obtained, including first information of a first driving scene and second information of a second driving scene. The number of obstacles in the second driving scene that differ from the first driving scene is less than a first threshold, and the distance between a second future trajectory of the vehicle in the second driving scene and the first future trajectory of the vehicle in the first driving scene is greater than a second threshold.

[0056] According to some embodiments, the first driving scenario is a historical driving scenario of the vehicle, and the second driving scenario is obtained by modifying obstacles in the first driving scenario. The first future trajectory is the actual future trajectory of the vehicle in the first driving scenario, and the second future trajectory is a simulated future trajectory obtained by trajectory planning based on the second driving scenario. By modifying obstacles in the historical driving scenario of the vehicle to generate a simulated driving scenario (i.e., the second driving scenario), and generating a first scenario sample pair based on the historical driving scenario and the corresponding simulated driving scenario, the sample generation efficiency can be improved, and the model's perception of key obstacles can be improved, thereby improving the model's training efficiency and accuracy.

[0057] According to some embodiments, a second scene sample can be obtained by the following steps: obtaining a first obstacle set for a first driving scene; modifying the first obstacle set to obtain a second obstacle set, wherein the number of obstacles in the second obstacle set that differ from the first obstacle set is less than a first threshold (e.g., 2); performing trajectory planning based on the second obstacle set to obtain a simulated future trajectory; and generating a second driving scene based on the second obstacle set in response to a distance between the simulated future trajectory and the real future trajectory being greater than a second threshold (e.g., 3 meters). Because the second obstacle set is obtained by modifying the first obstacle set, the number of modified obstacles is less than the first threshold, and the distance between the simulated future trajectory based on the second obstacle set and the real future trajectory based on the first obstacle set is greater than the second threshold, the obstacles that differ from the first obstacle set in the second obstacle set can be interpreted as critical obstacles that affect the driving trajectory. Generating the second driving scene based on this method and training a scene encoding model based on the first and second driving scenes can improve the scene encoding model's ability to perceive critical obstacles.

[0058] According to some embodiments, the first obstacle set is obtained by screening obstacles in the first driving scene based on decision tags. The specific steps are: obtaining a decision tag for each of at least one obstacle included in the first driving scene, the decision tag indicating whether the corresponding obstacle affects the vehicle's driving decision, and screening at least one obstacle based on the decision tag to generate the first obstacle set that affects the vehicle's driving decision. It will be understood that the decision tag is obtained by annotating the real trajectory of the first driving scene and is used to identify the vehicle's driving decision with respect to the obstacle. Decision tags include, for example, yield, overtake, and run over (e.g., paper, plastic bag, etc.). The first obstacle set selects obstacles from the first driving scene that have an impact on driving decisions, such as obstacles with decision tags of yield and overtake. By screening obstacles that affect driving decisions to form the first obstacle set, unnecessary calculations can be avoided, thereby reducing the amount of calculation and improving computational efficiency.

[0059] According to some embodiments, the modification operation of modifying the first obstacle set to obtain the second obstacle set includes at least one of the following:

[0060] Deleting obstacles, deleting obstacles in the first obstacle set whose number is less than a first threshold;

[0061] Add obstacles, adding obstacles whose number is less than the first threshold to the first obstacle set; or

[0062] Adjust the positions of the obstacles, and adjust the positions of obstacles in the first obstacle set whose number is less than the first threshold.

[0063] According to the above embodiment, the second driving scene can be flexibly generated through a variety of obstacle modification methods, thereby increasing the diversity of the first scene sample pairs and thus improving the generalization and robustness of the scene coding model.

[0064] Figure 3 A schematic diagram illustrating an obstacle modification method according to some embodiments of the present disclosure is shown. A first driving scene 310 (i.e., historical driving scene 310) is a real driving scene and includes obstacles 311, 312, 313, 314, and 315. A second driving scene (i.e., a simulated driving scene) can be generated by deleting obstacles from the first driving scene 310. For example, second driving scene 320 is generated by deleting obstacle 315 from the first driving scene 310. Second driving scene 320 includes obstacles 321, 322, 323, and 324. It will be appreciated that the only difference between second driving scene 320 and first driving scene 310 is the absence of obstacle 315. A second driving scene can also be generated by adding obstacles to the first driving scene 310. For example, second driving scene 330 is generated by adding obstacle 336 to the first driving scene 310. Second driving scene 330 includes obstacles 331, 332, 333, 334, 335, and 336. It is understood that the only difference between the second driving scene 330 and the first driving scene 310 is the added obstacle 336. The second driving scene can also be generated by adjusting the position of the obstacles in the first driving scene 310. For another example, the second driving scene 340 is obtained by adjusting the position of the obstacle 314 in the first driving scene 310. The second driving scene 340 includes obstacles 341, 342, 343, 344 and 345. It is understood that the only difference between the second driving scene 340 and the first driving scene 310 is the different positions of the obstacles 314 and 344. It is understood that Figure 3 The method in the figure is only exemplary, and the number of obstacles to be modified and the specific modification method can be selected according to actual conditions.

[0065] According to some embodiments, the first driving scene is a historical driving scene of the vehicle, and the real future trajectory of the vehicle in the first driving scene, i.e., the first future trajectory, can be obtained. The real future trajectory can be, for example, the real future trajectory within 6 seconds after the current frame of the first driving scene. By calculating the distance between the real future trajectory and the simulated future trajectory generated based on the second obstacle set, it can be determined whether the modified obstacles in the second obstacle set are obstacles that affect the driving trajectory. Based on the second obstacle set where the distance between the simulated future trajectory and the real future trajectory is greater than a second threshold, a second driving scene is generated. It can be understood that the obstacles in the second obstacle set of the second driving scene that are modified and cause the distance between the simulated future trajectory and the real future trajectory to be greater than the second threshold are obstacles that affect the driving trajectory and can be regarded as key obstacles.

[0066] Figure 4 A schematic diagram illustrating trajectory changes after obstacle modification according to an embodiment of the present disclosure is shown. Based on a first driving scenario 410, a corresponding real future trajectory 411 can be obtained. After removing obstacle 412 from first driving scenario 410, a simulated driving scenario 420 is generated. Through simulation, a simulated future trajectory 421 of simulated driving scenario 420 is obtained. It can be seen that the removal of obstacle 412 changes the vehicle's driving trajectory.

[0067] According to some embodiments, the distance between the simulated future trajectory and the real future trajectory may be the Euclidean distance between the two. The specific calculation method is not limited here.

[0068] In step S220, the first information is input into the scene coding model to obtain a first scene code output by the scene coding model.

[0069] In the present disclosure, any suitable model can be selected as a scene encoding model. For example, the Transformer model can be selected as the scene encoding model, and the encoder in the multipath++ model can also be selected as the scene encoding model. The Transformer model is based on the attention mechanism and can mine the interaction relationship between vehicles or between vehicles and obstacles in dynamic scenes. The encoder in the multipath++ model can use compact and efficient polylines to describe road features, and can directly use original state information (for example, position, speed, acceleration, etc.).

[0070] According to some embodiments, the first information of the first driving scene includes first obstacle information, road topology information (including information such as the number and location of lane lines, traffic light information, crosswalk information, stop line information, etc.). In addition, the real future trajectory corresponding to the first driving scene (for example, the real driving trajectory information within 6 seconds after the current frame of the first driving scene) can be obtained, wherein the first obstacle information includes the identification and historical information of each obstacle in the first obstacle set (for example, position, speed, acceleration information, etc.). The obstacles in the first obstacle set can be static obstacles or dynamic obstacles. Static obstacles include trees, fences, roadblocks, and buildings, while dynamic obstacles include motor vehicles, non-motor vehicles, and pedestrians. It is understandable that the information included in the first driving scene and the types of obstacles can be filtered according to actual conditions and are not limited here.

[0071] By inputting the first information into the scene coding model, a first scene code output by the scene coding model can be obtained.

[0072] In step S230, the second information is input into the scene coding model to obtain a second scene code output by the scene coding model.

[0073] The second information of the second driving scene includes a second obstacle set, road topology information (including information such as the number and location of lane lines, traffic light information, crosswalk information, stop line information, etc.). Furthermore, a simulated future trajectory (e.g., simulated driving trajectory information within 6 seconds of the current frame of the second driving scene) obtained through trajectory planning based on the second driving scene can be obtained. The second obstacle set includes obstacle identifiers and historical information (e.g., location, velocity, acceleration information, etc.).

[0074] The second information is input into the scene coding model to obtain a second scene code output by the scene coding model.

[0075] In embodiments of the present disclosure, the first and second information include different obstacle information. According to some embodiments, including the same road topology information in the first and second information not only enables the scene encoding model to identify key obstacles, but also enables the model to learn more scene information, thereby improving the model's accuracy.

[0076] In step S240 , a loss value of a scene coding model is determined based on a first similarity between the first scene coding and the second scene coding.

[0077] According to some embodiments, the first similarity is a cosine similarity between the first scene encoding and the second scene encoding, and a loss value of the scene encoding model can be determined based on the cosine similarity. The loss value of the scene encoding model can be calculated, for example, using the following loss function:

[0078]

[0079] Where N is the number of sample pairs of the first scene, v 1i is the first scene code of the i-th first scene sample pair, v 2i is the second scene code of the first scene sample pair i, Max() is the function for finding the maximum value, sim(v 1i ,v 2i ) is a first similarity (eg, cosine similarity) between the first scene encoding and the second scene encoding, and m1 is a preset constant.

[0080] According to an embodiment of the present disclosure, method 200 further includes: obtaining a second scene sample pair, wherein the second scene sample pair includes third information of a third driving scene, fourth information of a fourth driving scene, and a similarity label, wherein the similarity label indicates whether a third future trajectory of the vehicle in the third driving scene is similar to a fourth future trajectory of the vehicle in the fourth driving scene; inputting the third information into a scene coding model to obtain a third scene coding output by the scene coding model; and inputting the fourth information into the scene coding model to obtain a fourth scene coding output by the scene coding model. Determining a loss value of the scene coding model based on a first similarity between the first scene coding and the second scene coding includes: determining a loss value of the scene coding model based on the first similarity between the first scene coding and the second scene coding, a second similarity between the third scene coding and the fourth scene coding, and the similarity label.

[0081] According to some embodiments, the third driving scenario and the fourth driving scenario are both historical driving scenarios. The third future trajectory is the actual future trajectory of the vehicle in the third driving scenario, and the fourth future trajectory is the actual future trajectory of the vehicle in the fourth driving scenario. The scene encoding model is trained based on the second scene sample pairs generated from the two historical driving scenarios and the first scene sample pairs generated. This enables the model to accurately encode the entire scene and accurately represent key obstacles in the scene, thereby improving model accuracy.

[0082] The third information for the third driving scenario includes the obstacle set and road topology information (including the number and location of lane lines, traffic light information, crosswalk information, stop line information, etc.) in the driving scenario. Furthermore, the real future trajectory corresponding to the third driving scenario (e.g., the real driving trajectory information for the next 6 seconds after the current frame of the third driving scenario) can be obtained. The obstacle set includes obstacle identification and historical information (e.g., location, velocity, acceleration information, etc.).

[0083] The fourth information for the fourth driving scenario includes an obstacle set and road topology information (including the number and location of lane lines, traffic light information, crosswalk information, stop line information, etc.) in the driving scenario. Furthermore, the real future trajectory corresponding to the fourth driving scenario (e.g., real driving trajectory information within 6 seconds of the current frame of the fourth driving scenario) can be obtained. The obstacle set includes obstacle identifiers and historical information (e.g., location, velocity, acceleration information, etc.).

[0084] According to some embodiments, the distance between the real future trajectory corresponding to the third driving scenario and the real future trajectory corresponding to the fourth driving scenario is calculated to determine whether the two are similar, that is, to determine a similarity label for the two. For example, if the distance between the real future trajectories of the two is greater than a threshold (e.g., 2 meters), the similarity label indicates that the two are dissimilar; if the distance between the real future trajectories of the two is less than or equal to the threshold, the similarity label indicates that the two are similar. The third and fourth driving scenarios with similar real future trajectories can be determined as a scene positive sample pair, while the third and fourth driving scenarios with dissimilar real future trajectories can be determined as a scene negative sample pair.

[0085] According to some embodiments, the second similarity is a cosine similarity between the third scene encoding and the fourth scene encoding.

[0086] According to some embodiments, a loss value of a scene coding model may be determined based on a first similarity between a first scene coding and a second scene coding, a second similarity between a third scene coding and a fourth scene coding, and a similarity label. The loss value of the scene coding model may be calculated, for example, according to the following formula:

[0087]

[0088] In the above formula, N1 is the number of sample pairs of the first scene. 1i 、v 2i are the first scene code and the second scene code of the i-th first scene sample pair respectively. 1i ,v 2i ) is v 1i With v 2i The first similarity. Max() is the function that finds the maximum value. m1 is a preset constant. λ is the balance factor, which is a preset constant that represents the proportion of the first similarity in the total loss value.

[0089] N2 is the number of sample pairs of the second scene. 3i 、v 4i are the third scene code and the fourth scene code of the i-th second scene sample pair respectively. 3i ,v 4i ) is v 3iWith v 4i The second similarity. L i is the similarity label of the i-th second scene sample pair. In the case where the similarity label is “yes”, L i Can be set to 1, in the case where the similarity label is “no” L i Can be set to -1. m i is based on similarity label L i A constant to set. In the case where the similarity label is "yes", m i Can be set to 1, in the case where the similarity label is “no” m i Can be set to 0.

[0090] It should be noted that the constants in the above formula (such as m1, λ, m i ) can be set and changed according to the actual model training situation. According to some embodiments, during the model training process, the value of m1 can be set to the average second similarity of all second scene sample pairs with a similarity label of "yes" in the previous iteration minus a constant (e.g., 0.3).

[0091] According to some embodiments, N historical driving scenes of the vehicle may be obtained as first driving scenes. Based on the N first driving scenes, N second driving scenes may be generated. Each first driving scene is combined with the corresponding second driving scene to obtain N first scene sample pairs. Each first driving scene is combined with the other (N-1) first driving scenes to obtain (N 2 -N) pairs of second scene samples.

[0092] As described above, each second scene sample pair includes a third driving scene and a fourth driving scene. According to the above embodiment, the third driving scene can be any first driving scene, and the fourth driving scene can be any first driving scene that is not the third driving scene. A second scene sample pair (i.e., the third driving scene and the fourth driving scene) is generated based on any two driving scenes from the N first driving scenes, and a first scene sample pair (i.e., the first driving scene and the second driving scene) is generated based on each first driving scene from the N first driving scenes.

[0093] According to the above embodiment, the loss function in the above formula (2) can be replaced by:

[0094]

[0095] In the above formula, N is the number of the first driving scene. i is the first scene code of the i-th first driving scene, v i mockThe second scene code of the second driving scene corresponding to the i-th first driving scene. sim(v i ,v i mock ) is the first similarity. Max() is the function for finding the maximum value. m1 is a preset constant. λ is the balance factor, which is a preset constant that represents the proportion of the first similarity in the total loss value.

[0096] v j Encode the first scene of the jth first driving scene. sim(vi,v j ) is the second similarity. L ij is the similarity label between the i-th first driving scene and the j-th first driving scene. In the case where the similarity label is “yes”, L ij Can be set to 1, in the case where the similarity label is “no” L ij Can be set to -1. m ij is based on similarity label L ij A constant to set. In the case where the similarity label is "yes", m ij Can be set to 1, in the case where the similarity label is “no” m ij Can be set to 0.

[0097] It is understood that the constants in the above formula (such as m1, λ, m ij ) can be set and changed according to the actual model training situation. According to some embodiments, during the model training process, the value of m1 can be set to the average second similarity of all second scene sample pairs (i.e., positive sample pairs) with a similarity label of "yes" in the previous iteration minus a constant (e.g., 0.3).

[0098] It is understandable that the values ​​defined above can be set and changed according to the actual model training situation.

[0099] According to some embodiments, during the training of the scene coding model, the proportion of the first similarity in the loss value gradually decreases. That is, the value of λ is set relatively high in the early stages of model training, and the first similarity accounts for a relatively large proportion of the loss value, which can accelerate model convergence. As model training progresses, the value of λ gradually decreases, and the proportion of the first similarity in the loss value decreases, which can balance the relationship between the learning task of key obstacles and the learning task of the overall scene, thereby ensuring the training effect of the model.

[0100] In step S250, parameters of the scene coding model are adjusted based on the loss value.

[0101] According to an embodiment of the present disclosure, a trajectory planning method is also provided. Figure 5FIG. 5 shows a flow chart of a trajectory planning method 500 according to an embodiment of the present disclosure. The execution subject of each step of the method 500 may be a server (e.g. Figure 1 ), or a motor vehicle (e.g., a server 120 shown in FIG. Figure 1 110).

[0102] like Figure 5 As shown, the method 500 includes steps S510-S540.

[0103] In step S510, the current driving scene information of the vehicle is obtained.

[0104] In step S520, the driving scene information is input into a trained scene coding model to generate a current scene coding, wherein the scene coding model is trained based on the scene coding model training method 200 of the embodiment of the present disclosure.

[0105] In step S530 , similarities between the current scene code and a plurality of historical scene codes are calculated respectively.

[0106] In step S540, the planned trajectory of the vehicle in the current driving scenario is determined based on the future trajectory corresponding to the target historical scenario code, where the target historical scenario code is a historical scenario code having a similarity with the current scenario code greater than a threshold.

[0107] According to an embodiment of the present disclosure, the current scene is encoded using a scene coding model trained using a method based on an embodiment of the present disclosure, the similarity between the current scene coding and the historical scene coding is calculated, and trajectory planning is performed based on the similarity, which can improve the efficiency and accuracy of trajectory planning.

[0108] The following describes each step of method 500 in detail.

[0109] In step S510, the current driving scene information of the vehicle is obtained.

[0110] According to some embodiments, the vehicle's current driving scene information includes information about surrounding obstacles (including position, speed, acceleration information, etc.) and road topology information (including stop lines, lanes, crosswalks, etc.).

[0111] In step S520 , the driving scene information is input into the trained scene coding model to generate the current scene coding.

[0112] According to an embodiment of the present disclosure, the scene coding model is trained based on the scene coding model training method 200 of the embodiment of the present disclosure. Encoding the current scene using the scene coding model trained by method 200 can improve the accuracy of scene coding, thereby improving the accuracy of trajectory planning.

[0113] In step S530 , similarities between the current scene code and a plurality of historical scene codes are calculated respectively.

[0114] According to some embodiments, the similarity is the cosine similarity between the current scene encoding and the historical scene encoding. For example, the historical scene encoding can be the scene encoding of the first driving scene used when training the scene encoding model, and can be stored in advance on the server (for example, Figure 1 120) of a database (e.g., Figure 1 In the database 130 shown in ), there is no limitation here.

[0115] In step S540 , the planned trajectory of the vehicle in the current driving scenario is determined based on the future trajectory corresponding to the target historical scenario code.

[0116] According to an embodiment of the present disclosure, the target historical scene code is a historical scene code whose similarity to the current scene code is greater than a threshold value (eg, 0.9). The threshold value can be set according to actual needs and is not limited here.

[0117] According to some embodiments, there may be one or more target historical scene codes. If there is only one target historical scene code, the future trajectory corresponding to the target historical scene code may be used as the planned trajectory of the vehicle in the current driving scenario. Alternatively, the future trajectory corresponding to the target historical scene code may be adjusted (e.g., to avoid obstacles) and the adjusted trajectory may be used as the planned trajectory of the vehicle in the current driving scenario.

[0118] When there are multiple target historical scene codes, multiple future trajectories corresponding to the target historical scene codes can be obtained. According to some embodiments, collision detection can be performed on each future trajectory in descending order of similarity, that is, to determine whether the future trajectory will collide with surrounding obstacles. The first future trajectory that passes the collision detection (i.e., does not collide with surrounding obstacles) is used as the planned trajectory of the vehicle in the current driving scenario.

[0119] According to an embodiment of the present disclosure, a training device for a scene coding model is provided. Figure 6 FIG. 6 shows a structural block diagram of a training device 600 for a scene coding model according to an embodiment of the present disclosure. Figure 6 As shown, the apparatus 600 includes a first acquisition module 610 , a first input module 620 , a second input module 630 , a determination module 640 and an adjustment module 650 .

[0120] The first acquisition module 610 is configured to acquire a first scene sample pair, wherein the first scene sample pair includes first information of a first driving scene and second information of a second driving scene, the number of obstacles in the second driving scene that are different from the first driving scene is less than a first threshold, and a distance between a second future trajectory of the vehicle in the second driving scene and the first future trajectory of the vehicle in the first driving scene is greater than a second threshold.

[0121] The first input module 620 is configured to input the first information into the scene coding model to obtain a first scene code output by the scene coding model.

[0122] The second input module 630 is configured to input the second information into the scene coding model to obtain a second scene code output by the scene coding model.

[0123] The determination module 640 is configured to determine a loss value of the scene coding model based on a first similarity between the first scene coding and the second scene coding.

[0124] The adjustment module 650 is configured to adjust parameters of the scene coding model based on the loss value.

[0125] According to some embodiments, the first driving scenario is a historical driving scenario of the vehicle, and the second driving scenario is obtained by modifying obstacles in the first driving scenario; the first future trajectory is the real future trajectory of the vehicle in the first driving scenario, and the second future trajectory is a simulated future trajectory obtained by trajectory planning based on the second driving scenario.

[0126] According to some embodiments, the apparatus 600 further includes: a second acquisition module configured to acquire a first obstacle set of a first driving scene; a modification module configured to modify the first obstacle set to obtain a second obstacle set, wherein the number of obstacles that differ between the second obstacle set and the first obstacle set is less than a first threshold; a planning module configured to perform trajectory planning based on the second obstacle set to obtain a simulated future trajectory; and a generation module configured to generate a second driving scene based on the second obstacle set in response to a distance between the simulated future trajectory and the real future trajectory being greater than a second threshold.

[0127] According to some embodiments, the second acquisition module includes: an acquisition unit configured to obtain a decision label for each of at least one obstacle included in the first driving scene, the decision label indicating whether the corresponding obstacle affects the driving decision of the vehicle; and a generation unit configured to filter at least one obstacle based on the decision label to generate a first obstacle set that affects the driving decision of the vehicle.

[0128] According to some embodiments, the modification module may be configured to perform at least one of the following operations: deleting an obstacle; adding an obstacle; or adjusting a position of an obstacle.

[0129] According to some embodiments, the first information and the second information include the same road topology information.

[0130] According to some embodiments, the apparatus 600 further includes: a third acquisition module configured to acquire a second scene sample pair, wherein the second scene sample pair includes third information of a third driving scene, fourth information of a fourth driving scene, and a similarity tag, wherein the similarity tag indicates whether a third future trajectory of the vehicle in the third driving scene is similar to a fourth future trajectory of the vehicle in the fourth driving scene; a third input module configured to input the third information into a scene coding model to obtain a third scene code output by the scene coding model; and a fourth input module configured to input the fourth information into the scene coding model to obtain a fourth scene code output by the scene coding model. The determination module 640 is further configured to determine a loss value of the scene coding model based on a first similarity between the first scene code and the second scene code, a second similarity between the third scene code and the fourth scene code, and the similarity tag.

[0131] It should be understood that Figure 6 The modules or units of the apparatus 600 shown in FIG. 6 can be connected to Figure 2 The steps in the method 200 described above correspond to each other. Therefore, the operations, features and advantages described in the method 200 are also applicable to the device 600 and the modules and units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0132] According to an embodiment of the present disclosure, a trajectory planning device is provided. Figure 7 FIG. 7 shows a structural block diagram of a trajectory planning device 700 according to an embodiment of the present disclosure. Figure 7 As shown, the apparatus 700 includes an acquisition module 710 , a generation module 720 , a calculation module 730 and a determination module 740 .

[0133] The acquisition module 710 is configured to acquire the current driving scene information of the vehicle.

[0134] The generation module 720 is configured to input the driving scene information into a trained scene coding model to generate the current scene coding, wherein the scene coding model is trained by a training device based on the scene coding model of an embodiment of the present disclosure.

[0135] The calculation module 730 is configured to respectively calculate similarities between the current scene code and a plurality of historical scene codes.

[0136] The determination module 740 is configured to determine the planned trajectory of the vehicle in the current driving scenario based on the future trajectory corresponding to the target historical scenario code, wherein the target historical scenario code is a historical scenario code having a similarity with the current scenario code greater than a threshold.

[0137] It should be understood that Figure 7 The modules or units of the apparatus 700 shown in FIG. 7 can be connected to Figure 5 The steps in the method 500 described above correspond to each other. Therefore, the operations, features and advantages described in the method 500 are also applicable to the device 700 and the modules and units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0138] Although specific functionality is discussed above with reference to specific modules, it should be noted that the functionality of the various modules discussed herein may be separated into multiple modules, and / or at least some functionality of multiple modules may be combined into a single module.

[0139] It should also be understood that the various techniques described herein can be implemented in software, hardware, components or program modules. Figure 6 and Figure 7 The various modules described in the foregoing can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuits. For example, in some embodiments, one or more modules in modules 610-650 or modules 710-740 can be implemented together in a system on chip (SoC). SoC can include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and can optionally execute the received program code and / or include embedded firmware to perform functions.

[0140] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] According to an embodiment of the present disclosure, an electronic device is also provided, including: at least one processor; and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the scene coding model training method and trajectory planning method of the embodiment of the present disclosure.

[0142] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, and the computer instructions are used to enable a computer to execute the scene coding model training method and trajectory planning method of the embodiment of the present disclosure.

[0143] According to an embodiment of the present disclosure, a computer program product is further provided, comprising computer program instructions, which, when executed by a processor, implement the scene coding model training method and trajectory planning method of the embodiment of the present disclosure.

[0144] refer to Figure 8 , a block diagram of an electronic device 800 that can serve as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0145] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0146] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, an output unit 807, a storage unit 808, and a communication unit 809. The input unit 806 can be any type of device that can input information to the electronic device 800. The input unit 806 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone and / or a remote control. The output unit 807 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator and / or a printer. The storage unit 808 can include but is not limited to a magnetic disk, an optical disk. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as Bluetooth TM devices, 802.11 devices, Wi-Fi devices, WiMAX devices, cellular communication devices, and / or the like.

[0147] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as method 200 and method 500. For example, in some embodiments, method 200 and method 500 can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the method 200 and method 500 described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the method 200 and the method 500 in any other appropriate manner (eg, by means of firmware).

[0148] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0149] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0150] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0151] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0152] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0153] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0154] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0155] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present disclosure is not limited by these embodiments or examples, but is only limited by the claims after authorization and their equivalents. Various elements in the embodiments or examples can be omitted or replaced by their equivalents. In addition, the steps can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that as technology evolves, many of the elements described here can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A method for training a scene coding model, comprising: Obtaining a first scene sample pair, wherein the first scene sample pair includes first information of a first driving scene and second information of a second driving scene, the number of obstacles in the second driving scene that differ from the first driving scene is less than a first threshold, and a distance between a second future trajectory of the vehicle in the second driving scene and the first future trajectory of the vehicle in the first driving scene is greater than a second threshold; Inputting the first information into the scene coding model to obtain a first scene code output by the scene coding model; Inputting the second information into the scene coding model to obtain a second scene code output by the scene coding model; determining a loss value of the scene coding model based on a first similarity between the first scene coding and the second scene coding; and Parameters of the scene coding model are adjusted based on the loss value.

2. The method according to claim 1, wherein The first driving scene is a historical driving scene of the vehicle, and the second driving scene is obtained by modifying obstacles in the first driving scene; The first future trajectory is a real future trajectory of the vehicle in the first driving scenario, and the second future trajectory is a simulated future trajectory obtained by trajectory planning based on the second driving scenario.

3. The method according to claim 2, further comprising: Obtaining a first obstacle set for the first driving scene; Modifying the first obstacle set to obtain a second obstacle set, wherein the number of obstacles in the second obstacle set that are different from that in the first obstacle set is less than the first threshold; performing trajectory planning based on the second obstacle set to obtain the simulated future trajectory; and In response to a distance between the simulated future trajectory and the real future trajectory being greater than the second threshold, generating the second driving scenario based on the second obstacle set.

4. The method according to claim 3, wherein The acquiring of the first obstacle set of the first driving scene includes: Obtaining a decision label for each of at least one obstacle included in the first driving scene, the decision label indicating whether the corresponding obstacle affects the driving decision of the vehicle; and The at least one obstacle is screened based on the decision label to generate the first obstacle set that affects the driving decision of the vehicle.

5. The method according to claim 2, wherein: The modification includes at least one of the following: Remove obstacles; Add obstacles; or Adjust the position of the obstacle.

6. The method according to claim 1, wherein The first information and the second information include the same road topology information.

7. The method according to claim 1, further comprising: Obtaining a second scene sample pair, wherein the second scene sample pair includes third information of a third driving scene, fourth information of a fourth driving scene, and a similarity tag, wherein the similarity tag indicates whether a third future trajectory of the vehicle in the third driving scene is similar to a fourth future trajectory of the vehicle in the fourth driving scene; Inputting the third information into the scene coding model to obtain a third scene code output by the scene coding model; inputting the fourth information into the scene coding model to obtain a fourth scene code output by the scene coding model, The determining, based on the first similarity between the first scene encoding and the second scene encoding, a loss value of the scene encoding model includes: A loss value of the scene coding model is determined based on a first similarity between the first scene coding and the second scene coding, a second similarity between the third scene coding and the fourth scene coding, and the similarity label.

8. The method according to claim 7, wherein: During the training process of the scene coding model, the proportion of the first similarity in the loss value gradually decreases.

9. A trajectory planning method, comprising: Get the vehicle's current driving scene information; Inputting the driving scene information into a trained scene coding model to generate a current scene coding, wherein the scene coding model is trained based on the method of any one of claims 1 to 8; respectively calculating similarities between the current scene code and a plurality of historical scene codes; and Determining a planned trajectory of the vehicle in a current driving scenario based on a future trajectory corresponding to a target historical scenario code, wherein the target historical scenario code is a historical scenario code having a similarity with the current scenario code greater than a threshold.

10. A training device for a scene coding model, comprising: a first acquisition module configured to acquire a first scene sample pair, wherein the first scene sample pair includes first information of a first driving scene and second information of a second driving scene, the number of obstacles in the second driving scene that are different from the first driving scene is less than a first threshold, and a distance between a second future trajectory of the vehicle in the second driving scene and the first future trajectory of the vehicle in the first driving scene is greater than a second threshold; A first input module is configured to input the first information into the scene coding model to obtain a first scene code output by the scene coding model; A second input module is configured to input the second information into the scene coding model to obtain a second scene code output by the scene coding model; a determining module configured to determine a loss value of the scene coding model based on a first similarity between the first scene coding and the second scene coding; and An adjustment module is configured to adjust parameters of the scene coding model based on the loss value.

11. The device according to claim 10, wherein The first driving scene is a historical driving scene of the vehicle, and the second driving scene is obtained by modifying obstacles in the first driving scene; The first future trajectory is a real future trajectory of the vehicle in the first driving scenario, and the second future trajectory is a simulated future trajectory obtained by trajectory planning based on the second driving scenario.

12. The apparatus according to claim 11, further comprising: A second acquisition module is configured to acquire a first obstacle set of the first driving scene; a modification module configured to modify the first obstacle set to obtain a second obstacle set, wherein the number of obstacles in the second obstacle set that are different from that in the first obstacle set is less than the first threshold; a planning module configured to perform trajectory planning based on the second obstacle set to obtain the simulated future trajectory; and The generating module is configured to generate the second driving scenario based on the second obstacle set in response to a distance between the simulated future trajectory and the real future trajectory being greater than the second threshold.

13. The device according to claim 12, wherein The second acquisition module includes: an acquiring unit configured to acquire a decision label of each of at least one obstacle included in the first driving scene, the decision label indicating whether the corresponding obstacle affects the driving decision of the vehicle; and A generating unit is configured to screen the at least one obstacle based on the decision label to generate the first obstacle set that affects the driving decision of the vehicle.

14. The device according to claim 11, wherein The modification module is configured to perform at least one of the following operations: Remove obstacles; Add obstacles; or Adjust the position of the obstacle.

15. The device according to claim 10, wherein The first information and the second information include the same road topology information.

16. The apparatus according to claim 10, further comprising: a third acquisition module configured to acquire a second scene sample pair, wherein the second scene sample pair includes third information of a third driving scene, fourth information of a fourth driving scene, and a similarity tag, wherein the similarity tag indicates whether a third future trajectory of the vehicle in the third driving scene is similar to a fourth future trajectory of the vehicle in the fourth driving scene; a third input module configured to input the third information into the scene coding model to obtain a third scene code output by the scene coding model; a fourth input module configured to input the fourth information into the scene coding model to obtain a fourth scene code output by the scene coding model, Wherein, the determining module is further configured to: A loss value of the scene coding model is determined based on a first similarity between the first scene coding and the second scene coding, a second similarity between the third scene coding and the fourth scene coding, and the similarity label.

17. A trajectory planning device comprising: An acquisition module is configured to acquire current driving scene information of the vehicle; a generating module configured to input the driving scene information into a trained scene coding model to generate a current scene coding, wherein the scene coding model is trained based on the apparatus according to any one of claims 10 to 16; a calculation module configured to respectively calculate similarities between the current scene code and a plurality of historical scene codes; and The determination module is configured to determine the planned trajectory of the vehicle in the current driving scenario based on a future trajectory corresponding to a target historical scenario code, wherein the target historical scenario code is a historical scenario code having a similarity with the current scenario code greater than a threshold.

18. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

19. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 9.

20. A computer program product comprising computer program instructions, wherein: When the computer program instructions are executed by a processor, the method of any one of claims 1 to 9 is implemented.

Images