Action space dictionary generation method, intelligent device, and readable storage medium

Abstract

The application relates to the technical field of computers, in particular to a motion space dictionary generation method, an intelligent device and a readable storage medium, and aims to solve the technical problem of how to ensure that the generated motion space dictionary takes into account coverage, robustness and utilization. To this end, the application obtains trajectory data of multiple agents, performs grid division according to the trajectory data, obtains a grid map corresponding to the trajectory data, performs grid screening on each grid of the grid map, obtains a motion space dictionary according to the grid screening result, effectively improves the theoretical coverage of the motion space, effectively eliminates invalid grids, retains high-probability trajectories, effectively improves the utilization of the obtained motion space, and further effectively improves the robustness of the obtained motion space dictionary, thereby providing an effective data basis for scenarios such as model training, simulation and algorithm evaluation of automatic driving.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a method for generating an action space dictionary, an intelligent device, and a readable storage medium. Background Technology

[0002] In the field of autonomous driving trajectory generation, the action space dictionary (i.e., trajectory vocabulary) is a core component supporting the autoregressive generation paradigm. Its core function is to discretize the continuous multi-agent trajectories (i.e., corresponding "actions") in an autonomous driving scenario into a finite set of trajectory action units. This transforms the complex continuous trajectory distribution into a discrete space that can be efficiently modeled, enabling the generation and fitting of multimodal driving behaviors. Ultimately, this is applied to key scenarios such as data acquisition and algorithm evaluation in autonomous driving simulators.

[0003] Currently, the mainstream methods for constructing action space dictionaries in the industry can be divided into two categories: data-driven and rule-driven. However, both methods have significant technical shortcomings. Data-driven action space dictionaries have high utilization rates, but their robustness and coverage are insufficient. Rule-driven action space dictionaries have broad coverage, but their utilization rates are extremely low.

[0004] Accordingly, there is a need in this field for a new action space dictionary generation scheme to solve the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned deficiencies, this application is proposed to solve, or at least partially solve, the technical problem of how to ensure that the generated action space dictionary takes into account coverage, robustness and utilization.

[0006] In a first aspect, a method for generating an action space dictionary is provided, characterized in that the method includes:

[0007] Acquire trajectory data of multiple agents; the trajectory data includes multiple trajectories;

[0008] Based on the trajectory data, the data is divided into grids to generate a grid map corresponding to the trajectory data;

[0009] For each grid in the grid map, the grid is filtered based on the trajectory of the surrounding grids to obtain the grid filtering result;

[0010] The action space dictionary is obtained based on the grid corresponding to the grid filtering result and the trajectory data contained in the grid.

[0011] In one technical solution of the above-mentioned action space dictionary generation method, the step of performing grid division based on the trajectory data includes:

[0012] Based on the trajectory data, obtain the action set corresponding to each trajectory;

[0013] Based on the set of actions, a grid is created to obtain the gridded map corresponding to the trajectory data.

[0014] In one technical solution of the above-mentioned action space dictionary generation method, the step of obtaining the action set corresponding to each trajectory based on the trajectory data includes:

[0015] For each trajectory in the trajectory data, extract trajectory segments of equal length based on the trajectory, and transform the trajectory segments into a coordinate system centered on the agent;

[0016] In the coordinate system centered on the agent, the trajectory segment is differentiated to obtain the action of the trajectory segment at each moment, thereby obtaining the action set corresponding to the trajectory.

[0017] In one technical solution of the above-mentioned action space dictionary generation method, the step of performing gridded division based on the action set to obtain a gridded map corresponding to the trajectory data includes:

[0018] Construct a grid of preset size;

[0019] Based on the grid to which the endpoint of the trajectory corresponding to the action set belongs, the classification grid of the action set is determined, thereby obtaining a gridded map of the trajectory data.

[0020] In one technical solution of the above-mentioned action space dictionary generation method, the step of filtering the grid based on the trajectory conditions in the surrounding grids to obtain the grid filtering result includes:

[0021] For each grid of the grid map, if a trajectory exists in the grid, determine whether there are no trajectories in the surrounding grids of the grid.

[0022] If so, then the grid and the trajectory in the grid are removed.

[0023] In one technical solution of the above-mentioned action space dictionary generation method, the step of filtering the grid based on the trajectory conditions in the surrounding grids to obtain the grid filtering result includes:

[0024] For each grid of the grid map, if there is no trajectory in the grid, determine whether there is at least one grid with a trajectory in the surrounding grids of the grid.

[0025] If so, then retain the grid.

[0026] In one technical solution of the above-mentioned action space dictionary generation method, obtaining the action space dictionary based on the grid corresponding to the grid filtering result and the trajectory contained in the grid includes:

[0027] Based on the trajectories included in the grid corresponding to the grid filtering result, obtain the average trajectory of the grid;

[0028] Based on the average trajectory, obtain the yaw angle of the average trajectory corresponding to the grid;

[0029] The motion space dictionary is obtained based on the average trajectory and the yaw angle.

[0030] In one technical solution of the above action space dictionary generation method, obtaining the average trajectory of the grid based on the trajectory included in the grid corresponding to the grid filtering result includes:

[0031] Obtain the set of actions corresponding to the trajectories included in the grid;

[0032] Based on the set of actions, obtain the average trajectory of the grid.

[0033] In a second aspect, a smart device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the above-described methods for generating an action space dictionary.

[0034] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the method described in any of the technical solutions of the above-described action space dictionary generation method.

[0035] The above-described technical solutions of this application have at least one or more of the following beneficial effects:

[0036] In implementing the action space dictionary generation method provided in this application, this application acquires trajectory data of multiple agents, divides the trajectory data into grids to obtain a grid map corresponding to the trajectory data, performs grid filtering on each grid of the grid map, and obtains the action space dictionary based on the grid filtering results. Through the above configuration, this application can effectively improve the theoretical coverage of the action space for multi-agent trajectory data by dividing it into grids. Furthermore, by performing grid filtering based on the trajectory situation of the grids surrounding each grid, invalid grids can be effectively removed, and high-probability trajectories can be retained, thereby effectively improving the utilization rate of the obtained action space and thus effectively improving the robustness of the obtained action space dictionary. This provides an effective data foundation for autonomous driving model training, simulation, and algorithm evaluation scenarios. Attached Figure Description

[0037] The disclosure of this application will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Wherein:

[0038] Figure 1 This is a schematic flowchart of the main steps of an action space dictionary generation method according to an embodiment of this application;

[0039] Figure 2 This is a schematic flowchart of the main steps of an action space dictionary generation method according to one embodiment of the present application;

[0040] Figure 3 This is a schematic diagram illustrating the effect of a solution according to one embodiment of the present application;

[0041] Figure 4 This is a schematic diagram comparing the action space dictionary obtained by the action space dictionary generation method of this application with the prior art. Detailed Implementation

[0042] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0043] In the description of this application, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, and memory, and may also include software components, such as program code, or a combination of software and hardware. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular forms of the terms "a" and "this" can also include plural forms.

[0044] The relevant user personal information that may be involved in the various embodiments of this application is processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, based on the reasonable purpose of the business scenario, and includes personal information that users actively provide or that is generated as a result of using the product / service, as well as personal information obtained with user authorization.

[0045] The personal information processed in this application will vary depending on the specific product / service scenario and will be based on the specific scenario in which the user uses the product / service. This may involve the user's account information, device information, driving information, vehicle information, or other related information. This application will treat the user's personal information and its processing with the utmost diligence.

[0046] This application attaches great importance to the security of users' personal information and has taken reasonable and feasible security protection measures that comply with industry standards to protect users' information and prevent unauthorized access, disclosure, use, modification, damage or loss of personal information.

[0047] Here we will first explain some of the terms used in this application.

[0048] An action space dictionary (also known as a trajectory vocabulary) is a representation of a composite action space in reinforcement learning. It describes the multiple independent but combined action dimensions that an agent can execute at each step. It organizes multiple sub-action spaces (such as discrete and continuous) into a dictionary structure, which facilitates the modeling of complex control tasks.

[0049] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of an action space dictionary generation method according to an embodiment of this application. Figure 1 As shown, the action space dictionary generation method in this application embodiment mainly includes the following steps S101 to S104.

[0050] Step S101: Obtain trajectory data of multiple agents; the trajectory data includes multiple trajectories.

[0051] In this embodiment, multiple trajectory data of multiple agents in an autonomous driving scenario can be obtained.

[0052] In one implementation, the multiple agents may include vehicles, pedestrians, bicycles, etc. For example, the actual trajectories of vehicles, pedestrians, bicycles, and other agents can be obtained as trajectory data.

[0053] In one implementation, the trajectory data of the multi-agent can be recorded as [x,y,yaw] with a dimension of 3×T, where T is the number of trajectory data.

[0054] Step S102: Based on the trajectory data, perform grid division to generate a grid map corresponding to the trajectory data.

[0055] In this embodiment, the trajectory data can be divided into grids to generate a grid map of the trajectory data.

[0056] In one embodiment, step S102 may further include steps S1021 and S1022:

[0057] Step S1021: Based on the trajectory data, obtain the action set corresponding to each trajectory.

[0058] In this embodiment, for each trajectory in the trajectory data, trajectory segments of equal length can be extracted based on the trajectory, and these segments can be transformed into an agent-centric coordinate system. Within this agent-centric coordinate system, the trajectory segments are differentiated to obtain the action at each moment of the trajectory segment, thereby obtaining the action set corresponding to the trajectory.

[0059] Specifically, taking a trajectory segment [x, y, yaw] with a dimension of 3×T as an example, the trajectory segment is differentiated to obtain the coordinates at each time step, resulting in an action set [dx, dy, dyaw] with a dimension of 3x(T-1). Here, d is delta.

[0060] In one implementation, for each trajectory segment, a flip operation along the x-axis can be performed on the trajectory segment to obtain a mirror trajectory segment, and the original trajectory segment and the mirror trajectory segment can be merged to ensure the symmetry of the trajectory segment, thereby enhancing the robustness and representativeness of the subsequent action space.

[0061] Step S1022: Based on the action set, perform grid division and obtain the gridded map corresponding to the trajectory data.

[0062] In this embodiment, a grid of preset size can be constructed; based on the grid to which the endpoint of the trajectory corresponding to the action set belongs, the classification grid of the action set is determined, thereby obtaining a gridded map of the trajectory data.

[0063] Specifically, a fixed range and step size can be divided in a two-dimensional coordinate space to construct a network of a preset size, such as an H×W grid. The classification grid for an action set can be determined based on the grid to which the endpoint of a trajectory segment belongs. Here, H is the height of the grid, and W is the width of the grid.

[0064] Step S103: For each grid in the grid map, perform grid filtering based on the trajectory of the surrounding grids to obtain the grid filtering results.

[0065] In this embodiment, grid filtering can be performed on each grid in the grid map. That is, grid filtering can be achieved based on the trajectory of the surrounding grids.

[0066] In one embodiment, step S103 may further include the following steps S1031 and S1032:

[0067] Step S1031: For each grid of the grid map, if a trajectory exists in the grid, determine whether there is no trajectory in the surrounding grids; if so, proceed to step S1032.

[0068] Step S1032: Remove the grid and the trajectory in the grid.

[0069] In one embodiment, step S103 may further include the following steps S1033 and S1034:

[0070] Step S1033: For each grid of the grid map, if there is no trajectory in the grid, determine whether there is at least one grid with a trajectory in the surrounding grids; if so, proceed to step S1032.

[0071] Step S1034: Retain the grids where no trajectory exists.

[0072] Specifically, a large-scale analysis of real trajectories can be used to filter each grid. For each grid in the grid map, if its surrounding trajectory points are sparse, it can be considered noise and discarded. That is, it is determined whether there are any non-existent trajectories in the surrounding grids; if none exist, the grid can be discarded. For each grid in the grid map, if it has no trajectory of its own, but at least one of its surrounding grids contains a trajectory, the grid can be included; that is, the grid is considered valuable and set as "valid." Here, the surrounding grids are the eight neighboring grids adjacent to the current grid. This grid filtering method, which combines filtering and expansion, enhances the robustness of the grid filtering results against noise, thereby improving the coverage of the action space for trajectories that are not explicitly present. This eliminates invalid grids and retains high-probability grids, significantly improving the representativeness of the action space in the action space dictionary and thus increasing the utilization rate of the action space dictionary.

[0073] Step S104: Obtain the action space dictionary based on the grids corresponding to the grid selection results and the trajectory data contained in the grids.

[0074] In this embodiment, an action space dictionary can be constructed based on the grid and trajectory data corresponding to the grid filtering results.

[0075] In one implementation, the action space dictionary can be applied to scenarios such as autonomous driving simulation, closed-loop training, and algorithm evaluation.

[0076] In one embodiment, step S104 may further include steps S1041 to S1043:

[0077] Step S1041: Obtain the average trajectory of the grid based on the trajectories included in the grid corresponding to the grid filtering result.

[0078] In this embodiment, the average trajectory of a grid can be obtained based on the set of actions included in the trajectory of each grid in the grid filtering results. Specifically, for actions (dx, dy, dyaw) in a grid, the actions can be converted into trajectory points (dx, dy, sin(dyaw), cos(dyaw)) through calculation. Furthermore, the average trajectory can be obtained by averaging the trajectory points corresponding to all actions in the grid.

[0079] Step S1042: Obtain the yaw angle of the average trajectory corresponding to the grid based on the average trajectory.

[0080] In this embodiment, the yaw angle of the average trajectory in the grid can be obtained based on the average trajectory in the grid. Specifically, if the average trajectory in the grid is (dx, dy, sin(dyaw), cos(dyaw)), then the yaw angle of the average trajectory can be calculated based on the average trajectory. The yaw angle can be expressed as artan2(sin(dyaw), cos(dyaw)).

[0081] Step S1043: Obtain the motion space dictionary based on the average trajectory and yaw angle.

[0082] In this embodiment, an action space dictionary can be constructed based on the average trajectory and corresponding yaw angle of the grid preserved in the grid map.

[0083] Based on the methods described in steps S101 to S104 above, this embodiment of the application acquires trajectory data of multiple agents, divides the trajectory data into grids to obtain a grid map corresponding to the trajectory data, performs grid filtering on each grid of the grid map, and obtains an action space dictionary based on the grid filtering results. Through the above configuration, this embodiment of the application can effectively improve the theoretical coverage of the action space for multi-agent trajectory data by dividing it into grids. Furthermore, by performing grid filtering on the grids around each grid, invalid grids can be effectively removed, and high-probability trajectories can be retained, thereby effectively improving the utilization rate of the obtained action space and thus effectively improving the robustness of the obtained action space dictionary, thereby providing an effective data foundation for autonomous driving model training, simulation, and algorithm evaluation scenarios.

[0084] In one implementation, see Appendix Figure 2 , Figure 2 This is a schematic flowchart illustrating the main steps of an action space dictionary generation method according to one embodiment of this application. Figure 2 As shown, the action space dictionary generation method may include the following steps S201 to S206:

[0085] Step S201: Obtain trajectory data of the multi-agent system.

[0086] In this embodiment, the method described in step S201 is similar to that in step S101, and will not be repeated here for the sake of simplicity.

[0087] Step S202: Convert the trajectory data into an agent-centric coordinate system.

[0088] Step S203: The motion corresponding to the trajectory data is flipped along the x-axis.

[0089] In this embodiment, the methods described in steps S202 and S203 are similar to those in step S1021, and will not be repeated here for the sake of simplicity.

[0090] Step S204: Mesh the motion space.

[0091] In this embodiment, the method described in step S204 is similar to that in step S1022, and will not be described again here for the sake of simplicity.

[0092] Step S205: Filter outliers and expand their neighborhoods.

[0093] In this embodiment, the method described in step S205 is similar to that in step S103, and will not be repeated here for the sake of simplicity.

[0094] Step S206: Obtain the action space dictionary based on mesh fusion.

[0095] In this embodiment, the method described in step S206 is similar to that in step S104, and will not be repeated here for the sake of simplicity.

[0096] In a specific example, such as Figure 3 As shown, based on the action space dictionary generation method of this application, the data distribution in the data preparation stage is as follows: Figure 3 The first image shows the distribution after motion space flipping (flipping along the x-axis). Figure 3 The second image shows the distribution after gridding. Figure 3 The third image shows the distribution of outliers after filtering and expanding their neighborhoods. Figure 3 The fourth image shows the distribution of the mesh after fusion. Figure 3 As shown in the fifth image. From Figure 3 As can be seen from the embodiments of this application, the action space dictionary generation method can obtain an action space dictionary with higher coverage and more balanced distribution.

[0097] In a specific example, the action space dictionary generation method of this application embodiment can be compared with the prior art method of obtaining the action space dictionary using the K-disks algorithm. The action space dictionary generation method of this application embodiment has the advantages shown in Table 1 below:

[0098] Table 1. Comparison of the advantages of the action space dictionary generation method in this application.

[0099] k-disks (or k-discs) is a clustering algorithm extended from k-means, suitable for extracting "disk-shaped" regions with shape characteristics within clusters. The advantage of k-discs is its ability to identify clusters with subspace structure or boundaries, making it more expressive than k-means. However, it still relies on a data-driven clustering approach, is more susceptible to data noise, and is unsuitable for implicit trajectory combinations or overlays.

[0100] A comparative illustration of the visualization results of action space dictionaries within the same spatial range using k-disks and the action space dictionary generation method of this application is shown below. Figure 4 As shown. From Figure 4 As can be seen from the embodiments of this application, the action space dictionary generation method can effectively improve the trajectory symmetry of the action space dictionary, optimize the impact of noise, and effectively improve the coverage.

[0101] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effect of this application, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted solutions are equivalent to the technical solutions described in this application and therefore will also fall within the protection scope of this application.

[0102] Those skilled in the art will understand that all or part of the processes in the method of the above-described embodiment can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0103] Another aspect of this application provides a computer-readable storage medium.

[0104] In one embodiment of a computer-readable storage medium according to this application, the computer-readable storage medium can be configured to store a program that performs the action space dictionary generation method of the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described action space dictionary generation method. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium can be a storage device including various electronic devices, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0105] Another aspect of this application provides a smart device.

[0106] In one embodiment of a smart device according to this application, the smart device may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, which, when executed by the at least one processor, implements the method described in any of the embodiments of the above-described action space dictionary generation method. The smart device described in this application may include devices such as driving devices, smart vehicles, robots, and servers.

[0107] In one implementation, the memory and processor of the smart device can be connected via a bus for communication.

[0108] In some embodiments of this application, the processor may be a central processing unit, a microprocessor, a graphics processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor may be implemented in software, in hardware, or a combination of both.

[0109] The technical solution of this application has been described above with reference to one embodiment shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A method for generating an action space dictionary, characterized in that, The method includes: Acquire trajectory data of multiple agents; the trajectory data includes multiple trajectories; Based on the trajectory data, the data is divided into grids to generate a grid map corresponding to the trajectory data; For each grid in the grid map, the grid is filtered based on the trajectory of the surrounding grids to obtain the grid filtering result; Based on the grids corresponding to the grid filtering results and the trajectory data contained in the grids, the action space dictionary is obtained; The step of filtering the grid based on the trajectory of the surrounding grids to obtain the grid filtering result includes: For each grid of the grid map, if a trajectory exists in the grid, determine whether there are no trajectories in the surrounding grids of the grid. If so, then the grid and the trajectory in the grid are removed.

2. The action space dictionary generation method according to claim 1, characterized in that, The step of dividing the trajectory data into grids includes: Based on the trajectory data, obtain the action set corresponding to each trajectory; Based on the set of actions, a grid is created to obtain the gridded map corresponding to the trajectory data.

3. The action space dictionary generation method according to claim 2, characterized in that, The step of obtaining the action set corresponding to each trajectory based on the trajectory data includes: For each trajectory in the trajectory data, extract trajectory segments of equal length based on the trajectory, and transform the trajectory segments into a coordinate system centered on the agent; In the coordinate system centered on the agent, the trajectory segment is differentiated to obtain the action of the trajectory segment at each moment, thereby obtaining the action set corresponding to the trajectory.

4. The action space dictionary generation method according to claim 2, characterized in that, The step of dividing the action set into a grid and obtaining the gridded map corresponding to the trajectory data includes: Construct a grid of preset size; Based on the grid to which the endpoint of the trajectory corresponding to the action set belongs, the classification grid of the action set is determined, thereby obtaining a gridded map of the trajectory data.

5. The action space dictionary generation method according to claim 1, characterized in that, The step of filtering the grid based on the trajectory of the surrounding grids to obtain the grid filtering result includes: For each grid of the grid map, if there is no trajectory in the grid, determine whether there is at least one grid with a trajectory in the surrounding grids of the grid. If so, then retain the grid.

6. The action space dictionary generation method according to claim 1, characterized in that, The step of obtaining the action space dictionary based on the grid corresponding to the grid filtering result and the trajectory contained in the grid includes: Based on the trajectories included in the grid corresponding to the grid filtering result, obtain the average trajectory of the grid; Based on the average trajectory, obtain the yaw angle of the average trajectory corresponding to the grid; The motion space dictionary is obtained based on the average trajectory and the yaw angle.

7. The action space dictionary generation method according to claim 6, characterized in that, The step of obtaining the average trajectory of the grid based on the trajectories included in the grid corresponding to the grid filtering result includes: Obtain the set of actions corresponding to the trajectories included in the grid; Based on the set of actions, obtain the average trajectory of the grid.

8. A smart device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores a computer program, which, when executed by the at least one processor, implements the action space dictionary generation method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the action space dictionary generation method according to any one of claims 1 to 7.

Images