An unmanned vehicle scene reconstruction method, device, storage medium and terminal

Abstract

The application discloses an unmanned vehicle scene reconstruction method, which comprises the following steps: extracting a trajectory set of a dynamic entity in a vehicle flow video under a preset traffic scene; determining a dynamic entity to be analyzed in the vehicle flow video, constructing environment perception data and a first driving parameter set of the dynamic entity to be analyzed according to the trajectory set of the dynamic entity; inputting the scene electronic map and the environment perception data into an unmanned vehicle platform for simulation test, outputting a second driving parameter set of a target unmanned vehicle under the condition of the environment perception data; and determining behavior difference data of the target unmanned vehicle according to the first and second driving parameter sets. The application constructs environment perception data and a driving parameter set of a real vehicle through a vehicle flow video in a real scene, analyzes a driving parameter set of a virtual unmanned vehicle in combination with an unmanned vehicle platform, compares the driving parameters to quickly determine behavior difference data of the unmanned vehicle, and provides an improvement direction for improving the unmanned vehicle capability through the difference data.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to an autonomous vehicle scene reconstruction method, device, storage medium and terminal. Background Technology

[0002] Autonomous driving, as a future research direction for automobiles, has a profound impact on the automotive industry and even the transportation industry. The advent of driverless cars will free up human hands, reduce the frequency of traffic accidents, and ensure public safety. Simultaneously, with breakthroughs and continuous advancements in core technologies such as artificial intelligence and sensor detection, autonomous driving will inevitably become more intelligent and will also realize the industrialization of driverless vehicles. A scenario is a summary and expression of the types of driving environments encountered by autonomous vehicles during operation. Evaluating and training the driving capabilities of autonomous vehicles based on these scenarios is a necessary means to discover vulnerabilities in autonomous vehicle platforms and improve their environmental adaptability.

[0003] In existing technologies, although autonomous vehicles have undergone extensive scenario training in test fields, some complex and extreme traffic scenarios (high traffic volume, traffic accidents, pedestrian-vehicle mixed traffic at intersections, etc.) involve specific road patterns and a huge number of interactive objects, which are often impossible to simulate in the training field, but are indispensable in the training process of autonomous vehicles. Although some extreme scenarios can be generated by simulation methods, there is still a certain gap between them and the actual driving environment, which makes it impossible for autonomous vehicles to cope with various challenges in extreme scenarios in the actual environment. Summary of the Invention

[0004] This application provides an unmanned vehicle scene reconstruction method, apparatus, storage medium, and terminal. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0005] In a first aspect, embodiments of this application provide a method for reconstructing an unmanned vehicle scene, the method comprising:

[0006] Extract the trajectory set of dynamic entities from traffic flow videos under a preset traffic scenario;

[0007] Identify the dynamic entities to be analyzed in the traffic flow video, and construct the environmental perception data and the first set of driving parameters of the dynamic entities to be analyzed based on the trajectory set of the dynamic entities;

[0008] The electronic map data and environmental perception data of the preset traffic scenario are input into the unmanned vehicle platform for simulation testing, and the second set of driving parameters of the target unmanned vehicle under the environmental perception data conditions is output.

[0009] The driving capability of the target unmanned vehicle is evaluated based on the difference between the first set of driving parameters and the second set of driving parameters.

[0010] Optionally, before extracting the trajectory set of dynamic entities in the traffic flow video under a preset traffic scenario, the process also includes:

[0011] Identify the traffic scene to be recorded;

[0012] Plan the recording time period and spatial range of the traffic scenes to be recorded;

[0013] Plan the hovering coordinates and hovering altitude of the drone based on the spatial range;

[0014] The drone is controlled to record video based on the recording time period, hovering coordinates, and hovering altitude to obtain traffic flow video under the preset traffic scenario.

[0015] Optionally, extract the set of trajectories of dynamic entities in traffic flow videos under a preset traffic scenario, including:

[0016] Construct an ID identifier for each vehicle in the traffic flow video under a preset traffic scenario;

[0017] Determine the time range within which each vehicle appears in the traffic flow video;

[0018] Calculate the driving parameters of each vehicle at every moment within the time range;

[0019] Each vehicle's triple is constructed based on its ID, time range, and driving parameters to obtain the structured trajectory of each vehicle.

[0020] The trajectory set of dynamic entities is obtained based on the structured trajectory of each vehicle.

[0021] Optionally, calculate the driving parameters of each vehicle at each moment within the time range, including:

[0022] Obtain a rectangular area established based on the geodetic coordinate system of the UAV's surveillance range. The rectangular area includes the first geodetic coordinate system and the second geodetic coordinate system.

[0023] Obtain the envelope rectangle of each vehicle in the planar coordinate system within the UAV's field of view, and calculate the multiple boundary distances of the four sides of the envelope rectangle from the boundary of the planar coordinate system.

[0024] Calculate the geodetic coordinates, vehicle width, and vehicle length of each vehicle at each moment within the time range based on the first geodetic coordinates, the second geodetic coordinates, and multiple boundary distances.

[0025] Calculate the azimuth and instantaneous speed of each vehicle at every moment within the time range;

[0026] The driving parameters of each vehicle at each moment within the time range are determined as follows: the geodetic coordinates, vehicle width, vehicle length, azimuth angle, and instantaneous speed at each moment.

[0027] Optionally, environmental perception data and a first set of driving parameters for the dynamic entity to be analyzed are constructed based on the trajectory set of the dynamic entity, including:

[0028] Extract the vehicle trajectory of the dynamic entity to be analyzed from the trajectory set of the dynamic entity;

[0029] The target time range is determined from the vehicle trajectory of the dynamic entity to be analyzed;

[0030] Calculate the environmental perception data and the first set of driving parameters of the dynamic entity to be analyzed at each moment in the target time range.

[0031] Optionally, the environmental perception data and the first set of driving parameters of the dynamic entity to be analyzed at each moment in the target time range are calculated, including:

[0032] Extract the set of vehicle driving parameters appearing in the traffic flow video at each moment within the target time range from the trajectory set of dynamic entities;

[0033] Determine the first position at each moment within the target time range in the vehicle trajectory of the dynamic entity to be analyzed;

[0034] Construct a polygonal range centered on the first position to obtain the environmental perception range of the dynamic entity to be analyzed;

[0035] Obtain the second position of each of the vehicles in the vehicle driving parameter set, excluding the dynamic entity to be analyzed;

[0036] Spatial calculations are performed on each second location and the environmental perception range to determine whether each second location is within the environmental perception range. If so, environmental perception data of the dynamic entity to be analyzed at each moment in the target time range is generated.

[0037] The set of driving parameters of the dynamic entity to be analyzed at each moment within the target time range is determined as the first set of driving parameters of the dynamic entity to be analyzed.

[0038] Optionally, assessing the driving capability of the target unmanned vehicle based on the difference between the first set of driving parameters and the second set of driving parameters includes:

[0039] Calculate the similarity between the first set of driving parameters of the dynamic entity to be analyzed and the second set of driving parameters of the target unmanned vehicle;

[0040] Determine the behavioral differences of the target autonomous vehicle based on similarity.

[0041] Secondly, embodiments of this application provide an unmanned vehicle scene reconstruction device, the device comprising:

[0042] The trajectory set extraction module is used to extract the trajectory set of dynamic entities in traffic flow videos under preset traffic scenarios;

[0043] The data construction module is used to identify the dynamic entities to be analyzed in the traffic flow video, and to construct the environmental perception data and the first driving parameter set of the dynamic entities to be analyzed based on the trajectory set of the dynamic entities.

[0044] The simulation test module is used to input electronic map data and environmental perception data of a preset traffic scenario into the unmanned vehicle platform for simulation testing, and output the second set of driving parameters of the target unmanned vehicle under the environmental perception data conditions;

[0045] The parameter comparison and determination module is used to evaluate the driving capability of the target unmanned vehicle based on the difference between the first set of driving parameters and the second set of driving parameters.

[0046] Thirdly, embodiments of this application provide a computer storage medium storing multiple instructions adapted for loading and execution of the above-described method steps by a processor.

[0047] Fourthly, embodiments of this application provide a terminal that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed by the above-described method steps.

[0048] The technical solutions provided in this application embodiment may have the following beneficial effects:

[0049] In this embodiment, the unmanned vehicle scene reconstruction device first extracts the trajectory set of dynamic entities from traffic flow videos under a preset traffic scenario. Then, it identifies the dynamic entities to be analyzed in the traffic flow videos and constructs environmental perception data and a first set of driving parameters for these entities based on their trajectory sets. Next, it inputs the scene's electronic map and environmental perception data into an unmanned vehicle platform for simulation testing, outputting a second set of driving parameters for the target unmanned vehicle under the conditions of the environmental perception data. Finally, it determines the behavioral difference data of the target unmanned vehicle based on the first and second sets of driving parameters. Since this application constructs environmental perception data and driving parameter sets of real vehicles from traffic flow videos in real-world scenarios, and combines this with the analysis of virtual unmanned vehicle driving parameter sets by the unmanned vehicle platform, the behavioral difference data of the unmanned vehicle can be quickly determined by comparing the driving parameters. This difference data can then provide improvement directions for enhancing the capabilities of the unmanned vehicle. Furthermore, theoretically, each vehicle in the traffic flow video under a preset traffic scenario can be used as a dynamic entity to be analyzed, from which environmental perception data and a first set of driving parameters can be extracted for training and testing by the unmanned vehicle platform. This can rapidly expand the training and testing set of the unmanned vehicle in extreme scenarios, which is of great significance for improving the capabilities of unmanned vehicles.

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0052] Figure 1 This is a flowchart illustrating an unmanned vehicle scene reconstruction method provided in an embodiment of this application;

[0053] Figure 2 This is a schematic diagram illustrating the acquisition of video data of complex scenes based on UAV aerial photography, as provided in an embodiment of this application.

[0054] Figure 3 This is a schematic diagram illustrating a video structuring implementation for trajectory tracking of a single vehicle, provided in an embodiment of this application.

[0055] Figure 4 This is a schematic diagram illustrating the conversion from planar coordinates to geodetic coordinates provided in an embodiment of this application;

[0056] Figure 5 This is a schematic diagram of environmental perception data of an unmanned vehicle at a certain moment, provided in an embodiment of this application.

[0057] Figure 6 This is a schematic diagram of the structure of an unmanned vehicle scene reconstruction device provided in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0059] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them.

[0060] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0061] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0062] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0063] This application provides a method, apparatus, storage medium, and terminal for reconstructing unmanned vehicle scenes to address the problems existing in the aforementioned related technologies. In the technical solution provided by this application, environmental perception data and driving parameter sets of real vehicles are constructed from traffic flow videos in real-world scenarios. These are then combined with the driving parameter sets of a virtual unmanned vehicle analyzed by an unmanned vehicle platform. By comparing the driving parameters, behavioral difference data of the unmanned vehicle can be quickly determined, and this difference data can provide improvement directions for enhancing the capabilities of the unmanned vehicle. The following detailed description uses exemplary embodiments.

[0064] The following will be combined with the appendix Figure 1 -Appendix Figure 5This application provides a detailed description of the unmanned vehicle scene reconstruction method provided in its embodiments. This method can be implemented using a computer program and can run on an unmanned vehicle scene reconstruction device based on the von Neumann architecture. This computer program can be integrated into an application or run as a standalone utility application.

[0065] Please see Figure 1 This is a flowchart illustrating a method for reconstructing an unmanned vehicle scene, as provided in this application embodiment. Figure 1 As shown, the method in this application embodiment may include the following steps:

[0066] S101, Extract the set of trajectories of dynamic entities in the traffic flow video under the preset traffic scenario;

[0067] The preset traffic scenario is a traffic flow scenario at a specific location on the traffic network that the user has pre-defined. This can be a regular traffic scenario or a complex or extreme one, such as heavy traffic, traffic accidents, or mixed pedestrian and vehicle traffic at intersections. Dynamic entities include vehicles, pedestrians, and non-motorized vehicles, etc. The following text will focus on vehicles as an example.

[0068] In this embodiment, before extracting the trajectory set of dynamic entities from the traffic flow video under a preset traffic scenario, it is necessary to use a drone to record traffic flow video at a certain location on a user-preset traffic network. First, the traffic scenario to be recorded is determined, then the recording time period and spatial range of the traffic scenario to be recorded are planned, then the hovering coordinates and hovering height of the drone are planned according to the spatial range, and finally, the drone is controlled to record video based on the recording time period, hovering coordinates, and hovering height to obtain the traffic flow video under the preset traffic scenario.

[0069] Specifically, unmanned aerial vehicles (UAVs) have significant advantages in recording complex traffic scenarios. On the one hand, UAVs possess unique flexibility and maneuverability, facilitating dynamic observation and coverage of complex scenes. On the other hand, UAVs can be manually controlled to hover and have wide-angle shooting capabilities, enabling the acquisition of more comprehensive and clearer video data. Therefore, this application utilizes UAVs to record video of complex scenes and generates driving environment data for unmanned vehicles in these scenes. This is of great significance for the capability assessment and training of unmanned vehicles.

[0070] In one embodiment, a complex traffic scenario is first identified, and the recording time period T (let's call it T1 to T2) and spatial range S (let's call it a rectangle defined from the lower left corner (X0, Y0) to the upper right corner (X1, Y1)) are planned. Here, the aspect ratio of S is specified to match the aspect ratio of the UAV's field of view. High-precision map data within the spatial range S is acquired in advance. Then, the UAV's hovering coordinates (X, Y) and hovering altitude H are planned based on the spatial range. Finally, the UAV's flight path is planned according to the set parameters. The UAV, equipped with a high-resolution camera, follows the planned flight path, for example, entering the location at coordinates (X, Y) and hovering at altitude H at time T1, and begins recording video until time T2, recording the traffic flow video of the complex traffic scenario.

[0071] like Figure 2 As shown, in this application, the drone uses a vertical photography method for video recording, with the lens perpendicular to the ground. Therefore, the drone's hovering coordinates are... The drone's hovering altitude H should be such that the drone's field of view can just cover the spatial range S of the complex scene, thereby obtaining the clearest traffic flow video.

[0072] In this embodiment, after obtaining the traffic flow video under a preset traffic scenario, the trajectory set of dynamic entities in the traffic flow video under the preset traffic scenario can be extracted. First, the ID identifier of each vehicle in the traffic flow video under the preset traffic scenario is constructed. Then, the time range in which each vehicle appears within the traffic flow video range is determined. Next, the driving parameters of each vehicle at each moment within the time range are calculated. Then, a triplet of each vehicle is constructed based on the ID identifier, time range, and driving parameters to obtain the structured trajectory of each vehicle. Finally, the trajectory set of dynamic entities is obtained based on the structured trajectory of each vehicle.

[0073] In the scenario described in this application, by processing UAV video, such as using the method described in the paper "Vehicle tracking by detection in UAV aerial video," trajectory tracking of a single vehicle can be achieved. Figure 3 (A) shows the trajectory tracking effect of vehicles at the intersection at different times. Figure 3 (B) is a schematic diagram of the video structuring result at time t3 (A). As can be seen from the figure, the algorithm assigns a unique ID to each moving vehicle and gives the envelope rectangle of each vehicle at time t3.

[0074] Furthermore, when calculating the driving parameters of each vehicle at each moment within the time range, firstly, a rectangular range is obtained based on the geodetic coordinate system of the UAV's monitoring range. The rectangular range includes the first geodetic coordinate system and the second geodetic coordinate system. Then, the envelope rectangle of each vehicle on the planar coordinate system within the UAV's field of view is obtained, and multiple boundary distances from the four sides of the envelope rectangle to the boundary of the planar coordinate system are calculated. Next, based on the first geodetic coordinate system, the second geodetic coordinate system, and the multiple boundary distances, the geodetic coordinate system, vehicle width, and vehicle length of each vehicle at each moment within the time range are calculated. Then, the azimuth angle and instantaneous speed of each vehicle at each moment within the time range are calculated. Finally, the driving parameters of each vehicle at each moment within the time range, as well as the geodetic coordinate system, vehicle width, vehicle length, azimuth angle, and instantaneous speed at each moment, are determined as the driving parameters of each vehicle at each moment within the time range.

[0075] Specifically, when recording video, the drone in this application uses a lens perpendicular to the ground. This avoids perspective calculations and coordinate transformations, allowing the vehicle's geodetic coordinates to be derived from its planar coordinates within the drone's field of view. Having acquired a high-precision map of a complex scene within a spatial range S, and due to the perpendicular shooting and small field of view, this application assumes that within the spatial range S, the changes in geodetic coordinates and planar coordinates are linearly related, such as... Figure 3 As shown in (C), for a given rectangular area defined by geodetic coordinates (X0, Y0) to (X1, Y1), the vehicle's coordinates in the geodetic coordinate system can be calculated proportionally, as follows: Figure 4 As shown.

[0076] Figure 4 (A) shows the planar coordinate system within the UAV's field of view. Figure 4 (B) shows the geodetic coordinate system of the UAV's surveillance range. Let r be the envelope rectangle of a vehicle identified by the algorithm, and let the distances of the four sides of the envelope rectangle from the boundary of the plane coordinate system be a1, a2, b1, and b2, respectively. In this application, the center of the rectangle represents the vehicle's position. Then, based on the scale, the coordinates of the UAV in the geodetic coordinate system can be calculated as follows:

[0077]

[0078] Similarly, we can conclude that:

[0079]

[0080] Furthermore, the width w and height h of the rectangle can be obtained as follows:

[0081] Where A and B are the width and height of the planar coordinate system enclosed by the UAV's field of view, respectively, and the function dis(,) takes the distance between the two coordinate points in the geodetic coordinate system.

[0082] It should be noted that, since the object envelope rectangle of the video recognition algorithm is parallel to the field of view boundary, the vehicle's envelope rectangle only coincides with the vehicle's appearance when the vehicle is traveling in the positive direction (due south or due north, or due east or due west). Therefore, this invention uses the envelope rectangle when the vehicle is traveling in the positive direction to extract the length and width of the vehicle.

[0083] Through the above operations, any car c in the traffic flow video can be extracted. i In t j The geodetic coordinates of the time are denoted as The video capture frequency is 24 frames per second. Generally speaking, the refresh frequency of environmental perception data of autonomous vehicles is higher than that of video sampling rate. This invention uses linear interpolation to update the vehicle position to keep it the same as the refresh frequency of environmental perception data of autonomous vehicles.

[0084] For any car c in the video i This application uses triples This is used to represent the vehicle trajectories extracted after video structuring. Where, c i A vehicle is uniquely identified; (t) m ,t n The video provides the time range in which the vehicle appears within the video's field of view. The vehicle at t is given j The driving parameters at any given time include: geodetic coordinates. Vehicle C i Width and length (w) i ,h i ); Azimuth and instantaneous speed Depend on to Calculation of vector angles, Depend on to The average vehicle speed between intervals was calculated. The vehicle is given in (t) m ,t n A trajectory sequence within a time range, and having Where τ κ The refresh frequency (in Hz) for environmental perception data of autonomous vehicles.

[0085] S102, identify the dynamic entities to be analyzed in the traffic flow video, and construct the environmental perception data and the first driving parameter set of the dynamic entities to be analyzed based on the trajectory set of the dynamic entities;

[0086] In this embodiment of the application, the dynamic entity to be analyzed in the traffic video is first determined, then the vehicle trajectory of the dynamic entity to be analyzed is extracted from the trajectory set of the dynamic entity, then the target time range is determined from the vehicle trajectory of the dynamic entity to be analyzed, and finally the environmental perception data and the first driving parameter set of the dynamic entity to be analyzed at each moment in the target time range are calculated.

[0087] Specifically, when calculating the environmental perception data and the first set of driving parameters of the dynamic entity to be analyzed at each moment within the target time range, the driving parameter set of the vehicles appearing in the traffic flow video at each moment within the target time range is first extracted from the trajectory set of the dynamic entity. Then, the first position at each moment within the target time range is determined in the vehicle trajectory of the dynamic entity to be analyzed. Next, a polygon range centered on the first position is constructed to obtain the environmental perception range of the dynamic entity to be analyzed. Then, the second positions of the other vehicles besides the dynamic entity to be analyzed are obtained from the vehicle driving parameter set. Finally, each second position is spatially calculated with the environmental perception range to determine whether each second position is within the environmental perception range. If so, the environmental perception data of the dynamic entity to be analyzed at each moment within the target time range is generated, and the driving parameter set of the dynamic entity to be analyzed at each moment within the target time range is determined as the first set of driving parameters of the dynamic entity to be analyzed.

[0088] In one possible implementation, for the dynamic entity c to be analyzed i The time period appearing in the video (t) m ,t n ), which allows us to obtain the time t at each moment. j Environmental perception data Thus, we obtain the number of bicycles, c. i Environmental perception sequence set have Similarly, we can obtain the dynamic entity c to be analyzed. i First set of driving parameters

[0089] For example, after step S101, the trajectory set Ψ of dynamic entities in the traffic flow video can be obtained. For the dynamic entity c to be analyzed... i The time period appearing in the video (t) m ,t n Given a time t in ) J ∈(t m ,t n ), can extract all t in Ψ J The set of driving parameters of vehicles that are constantly visible in the video view is denoted as P. J ,have

[0090] Dynamic entity c to be analyzedi In t J The position of the moment is Azimuth and instantaneous speed Let the environmental perception range of the unmanned vehicle model κ to be tested be the dynamic entity c to be analyzed. i In t J Location at any moment The polygon range ρ constructed around the center k , to P J Except for c i Other vehicles (let's call them C) κ ) position With polygon ρ κ Perform spatial calculations and determine Is it in ρ κ If it is within the boundary, then it will Add to collection middle, The vehicles in the t can be considered as driverless cars. J Real-time environmental perception data.

[0091] like Figure 5 As shown, t J Vehicle C at all times i If the target vehicle is hidden, then its perceived environment contains 4 vehicles. The positions, vehicle boundaries, speeds, and steering angles of the four vehicles are given respectively; similarly... Vehicle c is given i The driving parameters.

[0092] S103, input the electronic map data and environmental perception data of the preset traffic scenario into the unmanned vehicle platform for simulation testing, and output the second set of driving parameters of the target unmanned vehicle under the environmental perception data conditions;

[0093] In one possible implementation, electronic map data of a pre-defined traffic scenario and the target vehicle c are used. i Environmental perception data P i The input is given to the autonomous vehicle platform, which then performs simulation testing. The result is set at (t). m ,t′ n The autonomous vehicle passed through the complex scene during the specified time period (due to the difference in behavior between the autonomous vehicle and the target vehicle, the time when the autonomous vehicle passed through the scene differed from that of the target vehicle), and the data was refreshed at the frequency τ according to the autonomous vehicle model κ. κ Output (t) m ,t′ n At each time point t′ within the time period j The driving parameters are denoted as This allows us to obtain the second set of driving parameters for the target unmanned vehicle.

[0094] S104, assess the driving capability of the target unmanned vehicle based on the difference between the first set of driving parameters and the second set of driving parameters.

[0095] In this embodiment of the application, when evaluating the driving capability of the target unmanned vehicle based on the difference between the first set of driving parameters and the second set of driving parameters, the driving parameters Ω′ of the target unmanned vehicle are compared. i and the dynamic entity c to be analyzed i Driving parameters Ω i It can find the similarity between the behavior of the target autonomous vehicle and the behavior of the human driver on the dynamic entity to be analyzed in the same environment.

[0096] Similarity can identify discrepancies, which can then be used to assess the capabilities of autonomous vehicles and provide directions for improvement.

[0097] In this embodiment of the application, when evaluating the driving ability of a vehicle based on the first set of driving parameters and the second set of driving parameters, the method further includes: extracting the interaction semantic data between the target unmanned vehicle and each dynamic entity based on the second set of driving parameters and the environmental perception data, wherein the interaction semantic data includes relative position, relative distance, and speed; comparing the relative position, relative distance, speed, and other data with driving specification data to evaluate the driving ability of the target unmanned vehicle.

[0098] It should be noted that this application can guarantee that the unmanned vehicle has the same driving environment as the dynamic entity being analyzed in complex scenario tests, but due to the unmanned vehicle's driving parameters Ω′ i Driving parameters Ω of human drivers i There are differences. At some point during the simulated driving process, the behavior of other vehicles in the perceived environment may become abnormally coupled with the behavior of the autonomous vehicle, or even cause an accident involving the autonomous vehicle. This is unrelated to the capabilities of the autonomous vehicle and should be considered during the testing phase.

[0099] In this embodiment, firstly, the trajectory set of dynamic entities in traffic flow videos under a preset traffic scenario is extracted. Then, the dynamic entities to be analyzed in the traffic flow videos are identified. Based on the trajectory set of the dynamic entities, environmental perception data and a first set of driving parameters for the dynamic entities to be analyzed are constructed. Next, the electronic map of the scenario and the environmental perception data are input into an autonomous vehicle platform for simulation testing. A second set of driving parameters for the target autonomous vehicle under the environmental perception data conditions is output. Finally, behavioral difference data of the target autonomous vehicle is determined based on the first and second sets of driving parameters. Since this application constructs environmental perception data and driving parameter sets of real vehicles from traffic flow videos in real-world scenarios, and combines this with the analysis of the driving parameter set of the virtual autonomous vehicle under the environmental perception data conditions by the autonomous vehicle platform, the behavioral difference data of the autonomous vehicle can be quickly determined by comparing the driving parameters. This difference data can then provide improvement directions for enhancing the capabilities of the autonomous vehicle.

[0100] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0101] Please see Figure 6 This illustration shows a structural schematic diagram of an unmanned vehicle scene reconstruction device provided in an exemplary embodiment of this application. This unmanned vehicle scene reconstruction device can be implemented as all or part of a terminal through software, hardware, or a combination of both. The device 1 includes a trajectory set extraction module 10, a data construction module 20, a simulation testing module 30, and a parameter comparison and judgment module 40.

[0102] The trajectory set extraction module 10 is used to extract the trajectory set of dynamic entities in the traffic flow video under the preset traffic scenario;

[0103] The data construction module 20 is used to identify the dynamic entities to be analyzed in the traffic flow video, and to construct the environmental perception data and the first driving parameter set of the dynamic entities to be analyzed based on the trajectory set of the dynamic entities.

[0104] The simulation test module 30 is used to input electronic map data and environmental perception data of a preset traffic scenario into the unmanned vehicle platform for simulation testing, and output the second set of driving parameters of the target unmanned vehicle under the environmental perception data conditions.

[0105] The parameter comparison and determination module 40 is used to evaluate the driving capability of the target unmanned vehicle based on the difference between the first set of driving parameters and the second set of driving parameters.

[0106] It should be noted that the autonomous vehicle scene reconstruction device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the autonomous vehicle scene reconstruction method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the autonomous vehicle scene reconstruction device and the autonomous vehicle scene reconstruction method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0107] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0108] In this embodiment, the unmanned vehicle scene reconstruction device first extracts the trajectory set of dynamic entities from traffic flow videos under a preset traffic scenario. Then, it identifies the dynamic entities to be analyzed in the traffic flow videos and constructs environmental perception data and a first set of driving parameters for these entities based on their trajectory sets. Next, it inputs the scene's electronic map and environmental perception data into an unmanned vehicle platform for simulation testing, outputting a second set of driving parameters for the target unmanned vehicle under the conditions of the environmental perception data. Finally, it determines the behavioral difference data of the target unmanned vehicle based on the first and second sets of driving parameters. Since this application constructs environmental perception data and driving parameter sets of real vehicles from traffic flow videos in real-world scenarios, and combines this with the analysis of virtual unmanned vehicle driving parameter sets by the unmanned vehicle platform, the behavioral difference data of the unmanned vehicle can be quickly determined by comparing the driving parameters. This difference data can then provide improvement directions for enhancing the capabilities of the unmanned vehicle. Furthermore, theoretically, each vehicle in the traffic flow video under a preset traffic scenario can be used as a dynamic entity to be analyzed, from which environmental perception data and a first set of driving parameters can be extracted for training and testing by the unmanned vehicle platform. This can rapidly expand the training and testing set of the unmanned vehicle in extreme scenarios, which is of great significance for improving the capabilities of unmanned vehicles.

[0109] This application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the unmanned vehicle scene reconstruction method provided in the above-described method embodiments.

[0110] This application also provides a computer program product containing instructions that, when run on a computer, causes the computer to execute the unmanned vehicle scene reconstruction method of the above-described method embodiments.

[0111] Please see Figure 7 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Figure 7 As shown, terminal 1000 may include: at least one processor 1001, at least one network interface 1004, user interface 1003, memory 1005, and at least one communication bus 1002.

[0112] The communication bus 1002 is used to realize the connection and communication between these components.

[0113] The user interface 1003 may include a display screen and a camera. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0114] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0115] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the electronic device 1000 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip, without being integrated into the processor 1001.

[0116] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 7 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an unmanned vehicle scene reconstruction application.

[0117] exist Figure 7 In the terminal 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and to obtain the user's input data; while the processor 1001 can be used to call the unmanned vehicle scene reconstruction application stored in the memory 1005 and specifically perform the following operations:

[0118] Extract the trajectory set of dynamic entities from traffic flow videos under a preset traffic scenario;

[0119] Identify the dynamic entities to be analyzed in the traffic flow video, and construct the environmental perception data and the first set of driving parameters of the dynamic entities to be analyzed based on the trajectory set of the dynamic entities;

[0120] The electronic map data and environmental perception data of the preset traffic scenario are input into the unmanned vehicle platform for simulation testing, and the second set of driving parameters of the target unmanned vehicle under the environmental perception data conditions is output.

[0121] The driving capability of the target unmanned vehicle is evaluated based on the difference between the first set of driving parameters and the second set of driving parameters.

[0122] In one embodiment, before the processor 1001 extracts the trajectory set of dynamic entities in the traffic video under a preset traffic scenario, it also performs the following operations:

[0123] Identify the traffic scene to be recorded;

[0124] Plan the recording time period and spatial range of the traffic scenes to be recorded;

[0125] Plan the hovering coordinates and hovering altitude of the drone based on the spatial range;

[0126] The drone is controlled to record video based on the recording time period, hovering coordinates, and hovering altitude to obtain traffic flow video under the preset traffic scenario.

[0127] In one embodiment, when the processor 1001 extracts the trajectory set of dynamic entities in a traffic flow video under a preset traffic scenario, it specifically performs the following operations:

[0128] Construct an ID identifier for each vehicle in the traffic flow video under a preset traffic scenario;

[0129] Determine the time range within which each vehicle appears in the traffic flow video;

[0130] Calculate the driving parameters of each vehicle at every moment within the time range;

[0131] Each vehicle's triple is constructed based on its ID, time range, and driving parameters to obtain the structured trajectory of each vehicle.

[0132] The trajectory set of dynamic entities is obtained based on the structured trajectory of each vehicle.

[0133] In one embodiment, when the processor 1001 performs the calculation of the driving parameters of each vehicle at each moment within the time range, it also performs the following operations:

[0134] Obtain a rectangular area established based on the geodetic coordinate system of the UAV's surveillance range. The rectangular area includes the first geodetic coordinate system and the second geodetic coordinate system.

[0135] Obtain the envelope rectangle of each vehicle in the planar coordinate system within the UAV's field of view, and calculate the multiple boundary distances of the four sides of the envelope rectangle from the boundary of the planar coordinate system.

[0136] Calculate the geodetic coordinates, vehicle width, and vehicle length of each vehicle at each moment within the time range based on the first geodetic coordinates, the second geodetic coordinates, and multiple boundary distances.

[0137] Calculate the azimuth and instantaneous speed of each vehicle at every moment within the time range;

[0138] The driving parameters of each vehicle at each moment within the time range are determined as follows: the geodetic coordinates, vehicle width, vehicle length, azimuth angle, and instantaneous speed at each moment.

[0139] In one embodiment, when the processor 1001 constructs environmental perception data and a first set of driving parameters for the dynamic entity to be analyzed based on the trajectory set of the dynamic entity, it specifically performs the following operations:

[0140] Extract the vehicle trajectory of the dynamic entity to be analyzed from the trajectory set of the dynamic entity;

[0141] The target time range is determined from the vehicle trajectory of the dynamic entity to be analyzed;

[0142] Calculate the environmental perception data and the first set of driving parameters of the dynamic entity to be analyzed at each moment in the target time range.

[0143] In one embodiment, when the processor 1001 executes the environmental perception data and the first set of driving parameters of the dynamic entity to be analyzed at each moment within the target time range, it specifically performs the following operations:

[0144] Extract the set of vehicle driving parameters appearing in the traffic flow video at each moment within the target time range from the trajectory set of dynamic entities;

[0145] Determine the first position at each moment within the target time range in the vehicle trajectory of the dynamic entity to be analyzed;

[0146] Construct a polygonal range centered on the first position to obtain the environmental perception range of the dynamic entity to be analyzed;

[0147] Obtain the second position of each of the vehicles in the vehicle driving parameter set, excluding the dynamic entity to be analyzed;

[0148] Spatial calculations are performed on each second location and the environmental perception range to determine whether each second location is within the environmental perception range. If so, environmental perception data of the dynamic entity to be analyzed at each moment in the target time range is generated.

[0149] The set of driving parameters of the dynamic entity to be analyzed at each moment within the target time range is determined as the first set of driving parameters of the dynamic entity to be analyzed.

[0150] In one embodiment, when the processor 1001 evaluates the driving capability of the target unmanned vehicle based on the difference between the first set of driving parameters and the second set of driving parameters, it specifically performs the following operations:

[0151] Calculate the similarity between the first set of driving parameters of the dynamic entity to be analyzed and the second set of driving parameters of the target unmanned vehicle;

[0152] Determine the behavioral differences of the target autonomous vehicle based on similarity.

[0153] In this embodiment, the unmanned vehicle scene reconstruction device first extracts the trajectory set of dynamic entities from traffic flow videos under a preset traffic scenario. Then, it identifies the dynamic entities to be analyzed in the traffic flow videos and constructs environmental perception data and a first set of driving parameters for these entities based on their trajectory sets. Next, it inputs the scene's electronic map and environmental perception data into an unmanned vehicle platform for simulation testing, outputting a second set of driving parameters for the target unmanned vehicle under the conditions of the environmental perception data. Finally, it determines the behavioral difference data of the target unmanned vehicle based on the first and second sets of driving parameters. Since this application constructs environmental perception data and driving parameter sets of real vehicles from traffic flow videos in real-world scenarios, and combines this with the analysis of virtual unmanned vehicle driving parameter sets by the unmanned vehicle platform, the behavioral difference data of the unmanned vehicle can be quickly determined by comparing the driving parameters. This difference data can then provide improvement directions for enhancing the capabilities of the unmanned vehicle. Furthermore, theoretically, each vehicle in the traffic flow video under a preset traffic scenario can be used as a dynamic entity to be analyzed, from which environmental perception data and a first set of driving parameters can be extracted for training and testing by the unmanned vehicle platform. This can rapidly expand the training and testing set of the unmanned vehicle in extreme scenarios, which is of great significance for improving the capabilities of unmanned vehicles.

[0154] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for reconstructing the unmanned vehicle scene can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0155] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for reconstructing a scene of an unmanned vehicle, characterized in that, The method includes: Extract the trajectory set of dynamic entities from traffic flow videos under a preset traffic scenario; Identify the dynamic entities to be analyzed in the traffic video, and construct the environmental perception data and the first driving parameter set of the dynamic entities to be analyzed based on the trajectory set of the dynamic entities; The electronic map data of the preset traffic scenario and the environmental perception data are input into the unmanned vehicle platform for simulation testing, and the second set of driving parameters of the target unmanned vehicle under the environmental perception data conditions is output. The driving capability of the target unmanned vehicle is evaluated based on the difference between the first set of driving parameters and the second set of driving parameters; The step of constructing the environmental perception data and the first driving parameter set of the dynamic entity to be analyzed based on the trajectory set of the dynamic entity includes: Extract the vehicle trajectory of the dynamic entity to be analyzed from the trajectory set of the dynamic entity; The target time range is determined from the vehicle trajectory of the dynamic entity to be analyzed; Calculate the environmental perception data and the first set of driving parameters of the dynamic entity to be analyzed at each moment in the target time range; The step of calculating the environmental perception data and the first set of driving parameters of the dynamic entity to be analyzed at each moment in the target time range includes: Extract the set of vehicle driving parameters appearing in the traffic video at each moment within the target time range from the trajectory set of the dynamic entity; The first position at each moment within the target time range is determined in the vehicle trajectory of the dynamic entity to be analyzed; Construct a polygonal range centered on the first position to obtain the environmental perception range of the dynamic entity to be analyzed; Obtain the second position of each of the other vehicles in the vehicle driving parameter set, excluding the dynamic entity to be analyzed; Spatial calculations are performed on each second location and the environmental perception range to determine whether each second location is within the environmental perception range. If so, environmental perception data of the dynamic entity to be analyzed at each moment in the target time range is generated. The set of driving parameters of the dynamic entity to be analyzed at each moment within the target time range is determined as the first set of driving parameters of the dynamic entity to be analyzed.

2. The method of claim 1, wherein, Before extracting the trajectory set of dynamic entities in the traffic flow video under the preset traffic scenario, the method further includes: Identify the traffic scene to be recorded; Plan the recording time period and spatial range of the traffic scene to be recorded; The hovering coordinates and hovering altitude of the UAV are planned based on the aforementioned spatial range; Based on the recording time period, hovering coordinates, and hovering height, the drone is controlled to record video to obtain traffic flow video under a preset traffic scenario.

3. The method of claim 1, wherein, The extraction of the trajectory set of dynamic entities in traffic flow videos under a preset traffic scenario includes: Construct an ID identifier for each vehicle in the traffic flow video under the preset traffic scenario; Determine the time range within which each vehicle appears in the traffic video; Calculate the driving parameters of each vehicle at each moment within the time range; Based on the ID, time range, and driving parameters of each vehicle, a triplet is constructed for each vehicle to obtain the structured trajectory of each vehicle. The trajectory set of dynamic entities is obtained based on the structured trajectory of each vehicle.

4. The method of claim 3, wherein, The calculation of the driving parameters of each vehicle at each moment within the time range includes: Obtain a rectangular area established based on the geodetic coordinate system of the UAV's surveillance range, wherein the rectangular area includes a first geodetic coordinate system and a second geodetic coordinate system; Obtain the envelope rectangle of each vehicle in the planar coordinate system within the UAV's field of view, and calculate multiple boundary distances of the four sides of the envelope rectangle from the boundary of the planar coordinate system. Calculate the geodetic coordinates, vehicle width, and vehicle length of each vehicle at each moment within the time range based on the first geodetic coordinates, the second geodetic coordinates, and multiple boundary distances; Calculate the azimuth and instantaneous speed of each vehicle at every moment within the stated time range; The driving parameters of each vehicle at each moment within the time range are determined as follows: the geodetic coordinates, vehicle width, vehicle length, azimuth angle, and instantaneous speed at each moment.

5. The method of claim 1, wherein, The step of evaluating the driving capability of the target unmanned vehicle based on the difference between the first set of driving parameters and the second set of driving parameters includes: Calculate the similarity between the first set of driving parameters of the dynamic entity to be analyzed and the second set of driving parameters of the target unmanned vehicle; The driving capability of the target autonomous vehicle is assessed based on the similarity.

6. A scene reconstruction device for unmanned vehicles, characterized in that, The device includes: The trajectory set extraction module is used to extract the trajectory set of dynamic entities in traffic flow videos under preset traffic scenarios; The data construction module is used to identify the dynamic entities to be analyzed in the traffic video, and to construct the environmental perception data and the first driving parameter set of the dynamic entities to be analyzed based on the trajectory set of the dynamic entities. The step of constructing the environmental perception data and the first driving parameter set of the dynamic entity to be analyzed based on the trajectory set of the dynamic entity includes: Extract the vehicle trajectory of the dynamic entity to be analyzed from the trajectory set of the dynamic entity; The target time range is determined from the vehicle trajectory of the dynamic entity to be analyzed; Calculate the environmental perception data and the first set of driving parameters of the dynamic entity to be analyzed at each moment in the target time range; The calculation of the environmental perception data and the first set of driving parameters of the dynamic entity to be analyzed at each moment in the target time range includes: Extract the set of vehicle driving parameters appearing in the traffic video at each moment within the target time range from the trajectory set of the dynamic entity; The first position at each moment within the target time range is determined in the vehicle trajectory of the dynamic entity to be analyzed; Construct a polygonal range centered on the first position to obtain the environmental perception range of the dynamic entity to be analyzed; Obtain the second position of each of the other vehicles in the vehicle driving parameter set, excluding the dynamic entity to be analyzed; Spatial calculations are performed on each second location and the environmental perception range to determine whether each second location is within the environmental perception range. If so, environmental perception data of the dynamic entity to be analyzed at each moment in the target time range is generated. The simulation test module is used to input the electronic map data of the preset traffic scenario and the environmental perception data into the unmanned vehicle platform for simulation testing, and output the second set of driving parameters of the target unmanned vehicle under the environmental perception data conditions; The parameter comparison and determination module is used to evaluate the driving capability of the target unmanned vehicle based on the difference between the first set of driving parameters and the second set of driving parameters.

7. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the method steps as claimed in any one of claims 1-5.

8. A terminal, characterized by comprising: include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the method steps as claimed in any one of claims 1-5.

Images