Automatic driving scene screening method and device, equipment and storage medium

Abstract

The present application relates to the field of automatic driving, and discloses an automatic driving scene screening method, device, equipment and storage medium, which are used for improving the efficiency and accuracy of scene screening. The method comprises the following steps: collecting road test data, and extracting the information of a host vehicle, a key area and a non-machine object from each video frame of the road test data; screening the video frame in which the host vehicle passes through the key area from the road test data according to the host vehicle information and the key area information, and obtaining an initial key area scene; identifying the non-machine object with a collision risk with the host vehicle in each frame of the initial key area scene, and determining the identified non-machine object as a candidate object; determining the distance between the candidate object and the host vehicle according to the host vehicle information and the candidate object information, recording the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, and obtaining at least one candidate key area scene; and removing the scene in which the host vehicle deviates from the original route from the at least one candidate key area scene.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to an autonomous driving scene screening method, device, equipment and storage medium. Background Art

[0002] The interaction between the driver and pedestrians and non-motorized vehicles is a crucial algorithmic scenario in autonomous driving. To develop and iterate more efficient human-vehicle interaction algorithms, it is necessary to select scenarios that meet the requirements for human-vehicle interaction from actual road test data for algorithm learning and testing.

[0003] Due to the complexity of the actual driving environment, in order to test different interaction scenarios in a targeted manner, it is necessary to accurately screen the road test data and sink the requirements into segmented interaction scenarios. The most common segmented interaction scenario is when the vehicle encounters pedestrians or non-motorized vehicles crossing the road while driving forward.

[0004] In existing technologies, it is usually necessary to manually identify and filter out scenarios that meet the requirements from road test data. However, the huge amount of data requires huge manpower and material resources, which is inefficient. Manual identification often has misjudgments and incorrect judgments caused by objective factors, and the accuracy of filtering the required scenarios is low. Summary of the Invention

[0005] The main purpose of the present invention is to provide an autonomous driving scene screening method, device, equipment and storage medium to solve the problem of not being able to quickly and accurately screen the required scenes, and to improve the efficiency and accuracy of screening.

[0006] A first aspect of the present invention provides a method for screening autonomous driving scenes, comprising: collecting drive test data, extracting main vehicle information, key area information, and non-motorized object information from each video frame of the drive test data, wherein the key area corresponding to the key area information is an area where non-motorized objects can move from one side of the road to the opposite side; screening video frames in which the main vehicle passes through the key area in the drive test data based on the main vehicle information and key area information in each video frame, to obtain at least one initial key area scene, wherein the initial key area scene is a sequence of video frames; identifying non-motorized objects that pose a risk of collision with the main vehicle in each frame of the at least one initial key area scene, determining the identified non-motorized objects as candidate objects, to obtain at least one candidate object and corresponding at least one candidate object information; determining the distance between the at least one candidate object and the main vehicle based on the main vehicle information and the at least one candidate object information, recording the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, to obtain at least one candidate key area scene; and removing scenes in which the main vehicle deviates from the original route from the at least one candidate key area scene.

[0007] Optionally, in a first implementation method of the first aspect of the present invention, the video frames in which the main vehicle passes through the key area in the road test data are filtered out according to the main vehicle information and key area information in each video frame to obtain at least one initial key area scene, including: reading the timestamp of each video frame in the road test data, and using the timestamp of each video frame as a reference, pushing forward a first preset time length and pushing backward a second preset time length to obtain a preset time period corresponding to each video frame; obtaining the main vehicle coordinates and key area range in each video frame according to the main vehicle information and key area information in each video frame; judging whether there is a video frame in each preset time period whose main vehicle coordinates are within the key area range, if so, retaining the video frame corresponding to the preset time period, and if not, removing the video frame corresponding to the preset time period; dividing the video frames retained in the road test data based on the continuity of the timestamps to obtain at least one video frame sequence, each video frame sequence being an initial key area scene, and obtaining at least one initial key area scene.

[0008] Optionally, in a second implementation of the first aspect of the present invention, the identifying of non-motorized objects that have a risk of collision with the main vehicle in each frame of the at least one initial key area scene, determining the identified non-motorized objects as candidate objects, and obtaining at least one candidate object and corresponding at least one candidate object information includes: identifying non-motorized objects within a preset distance range around the main vehicle in each frame of the at least one initial key area scene, the preset distance including a preset lateral distance and a preset longitudinal distance; obtaining a traveling speed of each identified non-motorized object, and calculating a direction angle between each identified non-motorized object and the main vehicle in the same video frame; when the traveling speed is greater than the preset speed and the direction angle is within the preset angle range, determining the corresponding non-motorized object as a candidate object, and obtaining at least one candidate object; obtaining at least one candidate object information corresponding to the at least one candidate object.

[0009] Optionally, in a third implementation manner of the first aspect of the present invention, the distance between the at least one candidate object and the main vehicle is determined based on the main vehicle information and the at least one candidate object information, the minimum distance in each video frame and the candidate object information corresponding to the minimum distance are recorded, and at least one candidate key area scene is obtained, including: reading at least one candidate video frame corresponding to the at least one candidate object, and obtaining the contour information of the main vehicle and the coordinate point of at least one candidate object in the at least one candidate video frame based on the main vehicle information and the at least one candidate object information; calculating the distance between each candidate object and the main vehicle in the same video frame based on the contour information of the main vehicle and the coordinate point of at least one candidate object in the at least one candidate video frame; recording the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, and obtaining at least one candidate key area scene.

[0010] Optionally, in a fourth implementation manner of the first aspect of the present invention, the calculating the distance between each candidate object and the main vehicle in the same video frame based on the contour information of the main vehicle in the at least one candidate video frame and the coordinate points of at least one candidate object includes: scanning the point on the contour of the main vehicle in each candidate video frame that is closest to the candidate object based on the contour information of the main vehicle in the at least one candidate video frame to obtain at least one candidate point, and the at least one candidate point corresponds one-to-one to the at least one candidate object; calculating the distance between the at least one candidate point and the coordinate point of the at least one candidate object to obtain at least one contour distance, and the at least one contour distance corresponds one-to-one to the at least one candidate object.

[0011] Optionally, in a fifth implementation manner of the first aspect of the present invention, removing the scene in which the main vehicle deviates from the original route from the at least one candidate key area scene includes: reading the timestamp of each video frame in the at least one candidate key area scene, and taking the timestamp of each video frame as a reference, pushing forward a third preset time length to obtain the first timestamp of each video frame, and pushing backward a fourth preset time length to obtain the second timestamp of each video frame; obtaining the main vehicle information in the first timestamp and the second timestamp corresponding to each video frame, determining the main vehicle information in the first timestamp as the first main vehicle information, and determining the main vehicle information in the second timestamp as the second main vehicle information; judging whether the main vehicle deviates from the original route based on the first main vehicle information and the second main vehicle information corresponding to each video frame, and if so, removing the candidate key area scene where the corresponding video frame is located.

[0012] Optionally, in a sixth implementation method of the first aspect of the present invention, the judgment of whether the main vehicle deviates from the original route based on the first main vehicle information and the second main vehicle information corresponding to each video frame, and if so, removing the candidate key area scene where the corresponding video frame is located, includes: obtaining the first orientation angle of the main vehicle in the first main vehicle information and the second orientation angle of the main vehicle in the second main vehicle information; calculating the angle between the first main vehicle orientation angle and the second main vehicle orientation angle; judging whether the angle is within a preset angle range, and if so, determining that the main vehicle deviates from the original route, and removing the candidate key area scene where the video frame corresponding to the angle is located.

[0013] A second aspect of the present invention provides an autonomous driving scene screening device, comprising: an acquisition module for collecting drive test data and extracting host vehicle information, key area information, and non-motorized object information from each video frame of the drive test data, wherein the key area corresponding to the key area information is an area where non-motorized objects can move from one side of the road to the opposite side; a screening module for screening video frames in the drive test data in which the host vehicle passes through the key area based on the host vehicle information and key area information in each video frame, thereby obtaining at least one initial key area scene, wherein the initial key area scene is a sequence of video frames; an identification module for identifying non-motorized objects that pose a collision risk with the host vehicle in each frame of the at least one initial key area scene, determining the identified non-motorized objects as candidate objects, thereby obtaining at least one candidate object and corresponding at least one candidate object information; a determination module for determining a distance between the at least one candidate object and the host vehicle based on the host vehicle information and the at least one candidate object information, recording the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, thereby obtaining at least one candidate key area scene; and a removal module for removing scenes in which the host vehicle deviates from its original route from the at least one candidate key area scene.

[0014] Optionally, in a first implementation method of the second aspect of the present invention, the screening module is specifically used to: read the timestamp of each video frame in the road test data, and based on the timestamp of each video frame, push forward a first preset time length and push backward a second preset time length to obtain a preset time period corresponding to each video frame; obtain the main vehicle coordinates and key area range in each video frame according to the main vehicle information and key area information in each video frame; determine whether there is a video frame in each preset time period whose main vehicle coordinates are within the key area, and if so, retain the video frame corresponding to the preset time period; if not, remove the video frame corresponding to the preset time period; based on the continuity of the timestamps, divide the video frames retained in the road test data to obtain at least one video frame sequence, each video frame sequence is an initial key area scene, and obtain at least one initial key area scene.

[0015] Optionally, in a second implementation of the second aspect of the present invention, the recognition module is specifically used to: identify non-motorized objects within a preset distance range around the main vehicle in each frame of the at least one initial key area scene, the preset distance including a preset lateral distance and a preset longitudinal distance; obtain the travel speed of each identified non-motorized object, and calculate the orientation angle between each identified non-motorized object and the main vehicle in the same video frame; when the travel speed is greater than the preset speed and the orientation angle is within the preset angle range, determine the corresponding non-motorized object as a candidate object, and obtain at least one candidate object; obtain at least one candidate object information corresponding to the at least one candidate object.

[0016] Optionally, in a third implementation of the second aspect of the present invention, the determination module includes: a first reading unit, used to read at least one candidate video frame corresponding to the at least one candidate object, and obtain the contour information of the main vehicle and the coordinate points of at least one candidate object in the at least one candidate video frame based on the main vehicle information and the at least one candidate object information; a calculation unit, used to calculate the distance between each candidate object and the main vehicle in the same video frame based on the contour information of the main vehicle in the at least one candidate video frame and the coordinate points of at least one candidate object; a recording unit, used to record the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, to obtain at least one candidate key area scene.

[0017] Optionally, in a fourth implementation manner of the second aspect of the present invention, the calculation unit is specifically used to: scan the point on the main vehicle contour in each candidate video frame that is closest to the candidate object based on the contour information of the main vehicle in the at least one candidate video frame to obtain at least one candidate point, and the at least one candidate point corresponds one-to-one to the at least one candidate object; calculate the distance between the at least one candidate point and the coordinate point of the at least one candidate object to obtain at least one contour distance, and the at least one contour distance corresponds one-to-one to the at least one candidate object.

[0018] Optionally, in a fifth implementation of the second aspect of the present invention, the removal module includes: a second reading unit, used to read the timestamp of each video frame in the at least one candidate key area scene, and based on the timestamp of each video frame, push forward a third preset time length to obtain the first timestamp of each video frame, and push backward a fourth preset time length to obtain the second timestamp of each video frame; a determination unit, used to obtain the main vehicle information in the first timestamp and the second timestamp corresponding to each video frame, determine the main vehicle information in the first timestamp as the first main vehicle information, and determine the main vehicle information in the second timestamp as the second main vehicle information; a removal unit, used to determine whether the main vehicle deviates from the original route based on the first main vehicle information and the second main vehicle information corresponding to each video frame, and if so, remove the candidate key area scene where the corresponding video frame is located.

[0019] Optionally, in a sixth implementation method of the second aspect of the present invention, the removal unit is specifically used to: obtain the first orientation angle of the main vehicle in the first main vehicle information and the second orientation angle of the main vehicle in the second main vehicle information; calculate the angle between the first main vehicle orientation angle and the second main vehicle orientation angle; determine whether the angle is within a preset angle range, and if so, determine that the main vehicle deviates from the original route, and remove the candidate key area scene where the video frame corresponding to the angle is located.

[0020] The third aspect of the present invention provides an autonomous driving scene screening device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the autonomous driving scene screening device executes the above-mentioned autonomous driving scene screening method.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium, which stores instructions that, when executed on a computer, enable the computer to execute the above-mentioned autonomous driving scenario screening method.

[0022] In the technical solution provided by the present invention, road test data is collected, and main vehicle information, key area information and non-motorized object information are extracted from each video frame of the road test data, wherein the key area corresponding to the key area information is an area where non-motorized objects can move from one side of the road to the opposite side; video frames in which the main vehicle passes through the key area in the road test data are filtered out based on the main vehicle information and key area information in each video frame to obtain at least one initial key area scene, which is a video frame sequence; non-motorized objects that have a collision risk with the main vehicle in each frame of the at least one initial key area scene are identified, and the identified non-motorized objects are determined as candidate objects to obtain at least one candidate object and corresponding at least one candidate object information; the distance between the at least one candidate object and the main vehicle is determined based on the main vehicle information and the at least one candidate object information, the minimum distance in each video frame and the candidate object information corresponding to the minimum distance are recorded to obtain at least one candidate key area scene; and scenes in which the main vehicle deviates from the original route are removed from the at least one candidate key area scene. In an embodiment of the present invention, video frames of a main vehicle passing through a key area are screened out according to preset conditions to obtain an initial scene. Non-motorized objects with the highest probability of collision with the main vehicle are identified in the initial scene to obtain candidate scenes. Other branch scenes in the candidate scenes are removed to obtain interactive scenes in which the main vehicle encounters a non-motorized object crossing the road while traveling forward, thereby improving screening efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of an embodiment of a method for screening autonomous driving scenarios in an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of key areas in an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of another embodiment of the method for screening autonomous driving scenarios in an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of the division of areas with collision risks in an embodiment of the present invention;

[0027] Figure 5 This is a schematic diagram of an embodiment of an autonomous driving scene screening device in an embodiment of the present invention;

[0028] Figure 6 This is a schematic diagram of another embodiment of the autonomous driving scene screening device in an embodiment of the present invention;

[0029] Figure 7 This is a schematic diagram of an embodiment of an autonomous driving scene screening device in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] Embodiments of the present invention provide a method, apparatus, device, and storage medium for screening autonomous driving scenarios. Key interaction scenarios in which a vehicle passes through key areas are selected from road test data based on preset conditions, thereby improving screening efficiency. At the same time, the distance between the vehicle and the nearest non-motorized object in the key interaction scenario is identified, which helps to classify the difficulty of the scenario. Unnecessary subdivisions in the key interaction scenario are then removed to obtain target scenarios that meet the requirements, thereby improving screening accuracy.

[0031] The terms "first," "second," "third," "fourth," and the like (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0032] To facilitate understanding, the following describes the specific process of an embodiment of the present invention. It is understood that the execution subject of the present invention can be an autonomous driving scene screening device, a terminal, or a server, and the specific implementation is not limited here. The embodiment of the present invention is described using a server as an example execution subject.

[0033] See also Figure 1 An embodiment of the autonomous driving scene screening method in an embodiment of the present invention includes:

[0034] 101. Collecting drive test data, extracting vehicle information, key area information, and non-motorized object information from each video frame of the drive test data, wherein the key area corresponding to the key area information is an area where a non-motorized object can move from one side of the road to the opposite side;

[0035] Multiple cameras are pre-installed on the main vehicle to capture the surrounding environment of the main vehicle. The server collects images taken by each camera at the same timestamp and identifies the element information in the image. The elements include roads, corners, road facilities, road signs, fixed obstacles, motor vehicles, pedestrians, non-motor vehicles, etc. The server generates road test data based on the information of each element and the main vehicle's own attributes. The road test data is a video that can observe the main vehicle and the surrounding environment of the main vehicle.

[0036] The server collects drive test data of a preset duration and extracts required information from the collected drive test data. Preferably, in order to improve the screening efficiency of effective scenarios, a time period in which the required scenario occurs more frequently within the preset duration is selected and the drive test data of the time period is collected.

[0037] For example, the server collects one month's road test data of the main vehicle, that is, the road test data from 00:00 on July 1, 2020 to 00:00 on August 1, 2020. Since the interaction scenarios between vehicles and non-motorized objects occur frequently during the morning and evening rush hours, the two time periods of 6:00-10:00 and 17:00-20:00 are selected. The server collects road test data from 6:00-10:00 and 17:00-20:00 every day between July 1, 2020 and August 1, 2020.

[0038] The server extracts the main vehicle information, key area information and non-motorized object information from each video frame of the road test data through the main vehicle's perception module and tracking algorithm. The main vehicle information includes the main vehicle identification, main vehicle coordinates, main vehicle orientation angle and main vehicle outline information; the key area information includes the key area range, and the key areas include zebra crossings, road aisles and intersections, such as Figure 2 As shown; the non-motorized object information includes the non-motorized object identification, non-motorized object category, non-motorized object coordinates, non-motorized object direction angle and non-motorized object running speed, among which the non-motorized object category is pedestrian, bicycle, electric vehicle or tricycle.

[0039] Furthermore, to reduce the amount of invalid data to be processed, the server only extracts the main vehicle information in each video frame and the key area information and non-motorized object information in the preset area around the main vehicle.

[0040] For example, the preset area is set to a circular area with the main vehicle as the center and a radius of 15 meters. The server extracts the main vehicle information in each video frame and the key area information and non-motorized object information within the 15-meter area around the main vehicle.

[0041] 102. Filter video frames in which the main vehicle passes through the key area in the drive test data based on the main vehicle information and key area information in each video frame to obtain at least one initial key area scene, where the initial key area scene is a video frame sequence;

[0042] The regular duration of time for motor vehicles to pass through the critical area is obtained based on historical data. Starting from the first frame of the road test data, based on the timestamp of each frame, it is determined whether there is a video frame of the main vehicle passing through the critical area within the regular duration. If so, the current video frame is filtered out. The main vehicle passing through the critical area means that the main vehicle body has entered the critical area. The method for determining whether the main vehicle has passed through the critical area is: select a point on the main vehicle as the reference point. When the coordinates of this point are within the range of the critical area, the server considers that the main vehicle has passed through the critical area. Usually, the center point of the main vehicle is selected as the reference point. Each video frame with a continuous time stamp in the filtered video frame sequence is determined as an initial key area scene.

[0043] For example, based on historical data, the typical duration for a motor vehicle to pass through a critical area is 10 seconds. A server acquires one hour of drive test data, and the camera captures video at 24 frames per second. Therefore, the drive test data contains 86,400 frames, covering 240 frames of the typical duration. Using the first frame of the drive test data as a reference, the 240 consecutive video frames are checked for any video frames showing the vehicle passing through the critical area. If not, the second frame of the drive test data is used as a reference to check for any video frames showing the vehicle passing through the critical area. If not, the third frame of the drive test data is used as a reference to check for any video frames showing the vehicle passing through the critical area. If so, the third frame is filtered out, and so on, until the filtering is complete. Among the filtered video frames, if two video frame sequences have consecutive timestamps, these two video frame sequences are identified as the two initial critical area scenes.

[0044] 103. Identify non-motorized objects that pose a collision risk with the host vehicle in each frame of at least one initial key area scene, determine the identified non-motorized objects as candidate objects, and obtain at least one candidate object and corresponding at least one candidate object information;

[0045] The server identifies non-motorized objects in each initial key area scene that pose a collision risk to the host vehicle. The identification is based on the following:

[0046] 1. The distance between the non-motorized object and the host vehicle is within the set range;

[0047] 2. The non-motorized object is in a moving state, that is, the running speed of the non-motorized object is greater than 0;

[0048] 3. A non-motorized object is in front of the main vehicle;

[0049] 4. The forward trajectory of the non-motorized object intersects with the forward trajectory of the main vehicle.

[0050] The identified non-motorized objects with collision risks are determined as candidate objects, and the non-motorized object identification, non-motorized object category, non-motorized object coordinates, non-motorized object direction angle and non-motorized object running speed corresponding to each candidate object are read to obtain the candidate object information.

[0051] 104. Determine the distance between the host vehicle and at least one candidate object based on the host vehicle information and the at least one candidate object information, record the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, and obtain at least one candidate key area scene;

[0052] The server scans the point on the main vehicle in each video frame where the candidate object is located that is closest to the candidate object, and calculates the straight-line distance between the point and the corresponding candidate object. If a video frame contains one straight-line distance, the straight-line distance and the corresponding candidate object information are recorded; if a video frame contains at least two straight-line distances, the minimum straight-line distance and the candidate object information corresponding to the minimum straight-line distance are recorded. The difficulty coefficient of the candidate key area scene can be evaluated based on the minimum distance contained in each candidate key area scene.

[0053] For example, there are seven candidate objects, among which candidate object 1 appears in video frame 1, candidate objects 2 and 3 appear in video frame 2, candidate objects 4, 5, and 6 appear in video frame 3, and candidate object 7 appears in video frame 4. The server scans point 1 on the main vehicle in video frame 1 that is closest to candidate object 1 and calculates the straight-line distance 1 between point 1 and candidate object 1; the server scans point 2 and point 3 on the main vehicle in video frame 2 that are respectively closest to candidate objects 2 and 3, and calculates the straight-line distance 2 between point 2 and candidate object 2, and the straight-line distance 3 between point 3 and candidate object 3; the server scans point 4, point 5, and point 6 on the main vehicle in video frame 3 that are respectively closest to candidate objects 4, 5, and 6, and calculates the straight-line distance 4 between point 4 and candidate object 4, the straight-line distance 5 between point 5 and candidate object 5, and the straight-line distance 6 between point 6 and candidate object 6; the server scans point 7 on the main vehicle in video frame 4 that is closest to candidate object 7 and calculates the straight-line distance 7 between point 7 and candidate object 7. Video frame 1 and video frame 4 have only one straight-line distance, namely straight-line distance 1 and straight-line distance 7. Therefore, straight-line distance 1 and information of candidate object 1 are recorded in video frame 1, and straight-line distance 7 and information of candidate object 7 are recorded in video frame 4. Video frame 2 has straight-line distance 2 and straight-line distance 3. After judgment, straight-line distance 2 is smaller than straight-line distance 3. Therefore, straight-line distance 2 and information of candidate object 2 are recorded in video frame 2. Video frame 3 has straight-line distance 4, straight-line distance 5 and straight-line distance 6. After judgment, straight-line distance 5 is smaller than straight-line distance 4 and straight-line distance 6. Therefore, straight-line distance 5 and information of candidate object 5 are recorded in video frame 3.

[0054] 105. Remove a scene in which the main vehicle deviates from the original route from at least one candidate key area scene.

[0055] A scenario in which the vehicle deviates from its original route refers to a scenario in which the vehicle makes a U-turn, turns left, or turns right near a key area. The server obtains the first vehicle heading angle when the vehicle approaches the key area and the second vehicle heading angle when the vehicle leaves the key area in the current candidate key area scenario. When the angle between the first and second vehicle heading angles is greater than a preset angle, the server determines that the vehicle has made a U-turn, turned left, or turned right, which is another subdivided human-vehicle interaction scenario. In this case, the current candidate key area scenario is removed from at least one candidate key area scenario.

[0056] For example, the vehicle heading angle when the main vehicle enters the key area is 0°. If the vehicle heading angle when the main vehicle leaves the key area and the vehicle heading angle when the main vehicle enters the key area in the candidate key area scene are greater than 45 degrees, it is considered that the main vehicle has turned left, turned right, or made a U-turn. At this time, the main vehicle has deviated from the original route, and this candidate key area scene is removed.

[0057] In an embodiment of the present invention, key interaction scenarios in which a vehicle passes through a key area are selected from road test data based on preset conditions, thereby improving screening efficiency. At the same time, identifying the distance between the vehicle and the nearest non-motorized object in the key interaction scenario helps to classify the difficulty of the scenario. Unnecessary subdivisions in the key interaction scenario are then removed to obtain interaction scenarios that meet the requirements, thereby improving screening accuracy.

[0058] See also Figure 3 Another embodiment of the method for screening autonomous driving scenarios in the embodiment of the present invention includes:

[0059] 201. Collecting drive test data, extracting vehicle information, key area information, and non-motorized object information from each video frame of the drive test data, wherein the key area corresponding to the key area information is an area where a non-motorized object can move from one side of the road to the opposite side;

[0060] Multiple cameras are pre-installed on the main vehicle to capture the surrounding environment of the main vehicle. The server collects images taken by each camera at the same timestamp and identifies the element information in the image. The elements include roads, corners, road facilities, road signs, fixed obstacles, motor vehicles, pedestrians, non-motor vehicles, etc. The server generates road test data based on the information of each element and the main vehicle's own attributes. The road test data is a video that can observe the main vehicle and the surrounding environment of the main vehicle.

[0061] The server collects drive test data of a preset duration and extracts required information from the collected drive test data. Preferably, in order to improve the screening efficiency of effective scenarios, a time period in which the required scenario occurs more frequently within the preset duration is selected and the drive test data of the time period is collected.

[0062] For example, the server collects one month's road test data of the main vehicle, that is, the road test data from 00:00 on July 1, 2020 to 00:00 on August 1, 2020. Since the interaction scenarios between vehicles and non-motorized objects occur frequently during the morning and evening rush hours, the two time periods of 6:00-10:00 and 17:00-20:00 are selected. The server collects road test data from 6:00-10:00 and 17:00-20:00 every day between July 1, 2020 and August 1, 2020.

[0063] The server extracts the main vehicle information, key area information and non-motorized object information from each video frame of the road test data through the main vehicle's perception module and tracking algorithm. The main vehicle information includes the main vehicle identification, main vehicle coordinates, main vehicle orientation angle and main vehicle outline information; the key area information includes the key area range, and the key areas include zebra crossings, road aisles and intersections, such as Figure 2 As shown; the non-motorized object information includes the non-motorized object identification, non-motorized object category, non-motorized object coordinates, non-motorized object direction angle and non-motorized object running speed, among which the non-motorized object category is pedestrian, bicycle, electric vehicle or tricycle.

[0064] Furthermore, to reduce the amount of invalid data to be processed, the server only extracts the main vehicle information in each video frame and the key area information and non-motorized object information in the preset area around the main vehicle.

[0065] For example, the preset area is set to a circular area with the main vehicle as the center and a radius of 15 meters. The server extracts the main vehicle information in each video frame and the key area information and non-motorized object information within the 15-meter area around the main vehicle.

[0066] 202. Filter video frames in which the main vehicle passes through the key area in the drive test data based on the main vehicle information and key area information in each video frame to obtain at least one initial key area scene, where the initial key area scene is a video frame sequence;

[0067] The server reads the timestamp of each video frame in the drive test data, and uses the timestamp of each video frame as a reference to advance forward by a first preset time length and backward by a second preset time length. If the time length after advancing to the first video frame of the drive test data is still less than the first preset time length, the time length between the first video frame and the current video frame is used as the first preset time length of the current video frame. Similarly, if the time length after advancing to the last video frame of the drive test data is still less than the second preset time length, the time length between the last video frame and the current video frame is used as the second preset time length of the current video frame. A preset time period corresponding to each video frame is obtained. The main vehicle coordinates and the key area range in each video frame are obtained based on the main vehicle information and key area information in each video frame. The main vehicle coordinates are the coordinates of the main vehicle center point. It is determined whether there is a video frame in each preset time period whose main vehicle coordinates are within the key area range. If so, the video frame corresponding to the preset time period is retained. If not, the video frame corresponding to the preset time period is removed. The video frames retained in the drive test data are divided based on the continuity of the timestamps to obtain at least one video frame sequence. Each video frame sequence is an initial key area scene, and at least one initial key area scene is obtained.

[0068] For example, if the first preset duration is set to 2 seconds and the second preset duration is set to 8 seconds, and the server acquires one hour of drive test data, and the video captured by the camera has 24 frames per second, then the drive test data contains 86,400 frames. The server reads the timestamp of video frame 1 and decelerates 8 seconds, or 192 frames, to obtain frame 193. It then checks whether any frame between frames 1 and 193 has the coordinates of the vehicle's center point within the critical area. If no such frame exists, frame 1 is removed. The server reads the timestamp of video frame 2 and decelerates 2 seconds forward and 8 seconds backward. Since the duration of frame 1 is less than 2 seconds, the duration between frame 2 and frame 1 is determined as the first duration of frame 2. It then decelerates 8 seconds backward, or 192 frames, to obtain frame 194. It then checks whether any frame between frames 2 and 194 has the coordinates of the vehicle's center point within the critical area. If no such frame exists, frame 2 is retained. This process repeats until all frames in the drive test data have been filtered.

[0069] As can be seen from the above example, during the screening process, video frames may be repeatedly detected. In order to improve data processing efficiency, the server adds a detection tag to the detected video frames during the screening process, and records the detection results in the detection tag. When the video frame is detected again, the server can read the detection results in the detection tag.

[0070] For example, during the first detection, the server detects whether the coordinates of the center point of the main vehicle in video frames 1 to 10 are within the key area. After detection, the coordinates of the center point of the main vehicle in video frames 1 to 7 are not within the key area, and the coordinates of the center point of the main vehicle in video frames 8 to 10 are within the key area. The server adds a detection identifier to video frames 1 to 10 and records the corresponding detection result in the detection identifier. During the second detection, the server needs to detect whether the coordinates of the center point of the main vehicle in video frames 2 to 11 are within the key area. Since video frames 2 to 10 have already been detected, the server directly reads the detection identifiers of video frames 2 to 10, and then continues to detect whether the coordinates of the center point of the main vehicle in video frame 11 are within the key area. After detection, the coordinates of the center point of the main vehicle in video frame 11 are within the key area. The server adds a detection identifier to video frame 11 and records the corresponding detection result in the detection identifier of video frame 11.

[0071] 203. Identify non-motorized objects that have a collision risk with the host vehicle in each frame of at least one initial key area scene, determine the identified non-motorized objects as candidate objects, and obtain at least one candidate object and corresponding at least one candidate object information;

[0072] The server identifies non-motorized objects within a preset distance range around the main vehicle in each frame of at least one initial key area scene, where the preset distance includes a preset lateral distance and a preset longitudinal distance. The server obtains the travel speed, heading angle, and heading angle of each identified non-motorized object and the main vehicle, calculates the heading angle between each identified non-motorized object and the main vehicle in the same video frame, and when the travel speed is greater than the preset speed and the heading angle is within the preset angle range, determines the corresponding non-motorized object as a candidate object, obtains at least one candidate object, records the non-motorized object information corresponding to the at least one candidate object, and obtains at least one candidate object information.

[0073] The method for calculating the orientation angle between the main vehicle and the non-motorized object is as follows: the server obtains the main vehicle orientation angle X and the non-motorized object orientation angle Y measured during the road test data collection process, calculates the value of YX, and when +180°≥YX≥-180°, the angle between the main vehicle and the non-motorized object is (YX)°; when YX>+180°, the angle between the main vehicle and the non-motorized object is (YX-360)°; when YX<-180°, the angle between the main vehicle and the non-motorized object is (Y-X+360)°.

[0074] The collision risk area around the main vehicle is divided based on the heading angle, preset lateral distance and preset longitudinal distance, such as Figure 4As shown, area one is the area in front of the left side of the main vehicle. In area one, if the speed of the non-motorized object is greater than the preset speed and the orientation angle between the non-motorized object and the main vehicle is within the first preset angle range, the corresponding non-motorized object is determined as a candidate object; area two is the area in front of the right side of the main vehicle. In area two, if the speed of the non-motorized object is greater than the preset speed and the orientation angle between the non-motorized object and the main vehicle is within the second preset angle range, the corresponding non-motorized object is determined as a candidate object; area three is the area between area one and area two directly in front of the main vehicle. In area three, if the speed of the non-motorized object is greater than the preset speed and the orientation angle between the non-motorized object and the main vehicle is within the third preset angle range, the corresponding non-motorized object is determined as a candidate object.

[0075] For example, area one is set as an area 2 to 4 meters away from the left side of the main vehicle and 0 to 15 meters away from the main vehicle in the longitudinal direction; area two is set as an area 2 to 4 meters away from the right side of the main vehicle and 0 to 15 meters away from the main vehicle in the longitudinal direction; area three is set as an area 2 meters away from the left side of the main vehicle, 2 meters away from the right side of the main vehicle, and 0 to 15 meters away from the main vehicle in the longitudinal direction. The preset speed is set to 0.2m / s, the server reads the travel speed of the non-motorized object, sets the first preset angle range to 45° to 135°, the second preset angle range to -135° to -45°, the third preset angle range to -135° to -45° and 45° to 135°, and when the non-motorized object is located in area one, the travel speed is greater than 0.2m / s, and the orientation angle between the non-motorized object and the main vehicle is within the range of 45° to 135°, the non-motorized object is confirmed as a candidate object; when the non-motorized object is located in area two, the travel speed is greater than 0.2m / s, and the orientation angle between the non-motorized object and the main vehicle is within the range of -135° to -45°, the non-motorized object is confirmed as a candidate object; when the non-motorized object is located in area three, the travel speed is greater than 0.2m / s, and the absolute value of the orientation angle between the non-motorized object and the main vehicle is within the range of 45° to 135°, the non-motorized object is confirmed as a candidate object.

[0076] 204. Calculate the distance between each candidate object and the main vehicle in the same video frame based on the at least one candidate object information and the main vehicle information;

[0077] The server reads at least one candidate video frame corresponding to at least one candidate object, obtains contour information of the main vehicle and coordinate points of at least one candidate object in the at least one candidate video frame based on the main vehicle information and the at least one candidate object information, scans the point on the main vehicle contour in each candidate video frame that is closest to the candidate object based on the contour information of the main vehicle in the at least one candidate video frame, obtains at least one candidate point, the at least one candidate point has a one-to-one correspondence with the at least one candidate object, calculates the distance between the at least one candidate point and the coordinate point of the at least one candidate object, obtains at least one contour distance, the at least one contour distance has a one-to-one correspondence with the at least one candidate object.

[0078] 205. Record the minimum distance in each video frame and the candidate object information corresponding to the minimum distance to obtain at least one candidate key area scene;

[0079] The server determines whether the video frame contains at least two calculation results. If so, it records the minimum distance and the candidate object information corresponding to the minimum distance. If not, it means that there is only one calculation result in the video frame. The server records the distance indicated by the calculation result and the corresponding candidate object information, and determines the initial key area scene with the distance and candidate object information recorded as the candidate key area scene, obtaining at least one candidate key area scene.

[0080] 206. Obtain a first timestamp and a second timestamp corresponding to each video frame in at least one candidate key area scene;

[0081] The server reads the timestamp of each video frame in at least one candidate key area scene, and uses the timestamp of each video frame as a reference, pushes forward a third preset time length to obtain a first timestamp of each video frame, and pushes backward a fourth preset time length to obtain a second timestamp of each video frame. The video frame corresponding to the first timestamp is the video frame of the main vehicle driving towards the key area, and the video frame corresponding to the second timestamp is the video frame of the main vehicle leaving the key area.

[0082] For example, read the video frame 1000 in at least one candidate key area scene, take the timestamp of the video frame 1000 as a reference, push forward 10 seconds to obtain the first timestamp of the video frame 1000, and push backward 8 seconds to obtain the second timestamp of the video frame 1000.

[0083] Furthermore, to ensure that the vehicle has not entered the critical area in the video frame corresponding to the first timestamp, thereby obtaining a more accurate initial orientation of the vehicle, the server retrieves the earliest video frame showing the vehicle passing through the critical area before the current video frame. The server then deflects the earliest video frame passing through the critical area forward by a third preset time duration to determine the first timestamp of the current video frame. If no video frame showing the vehicle passing through the critical area before the current video frame exists, the server then, following the original rule, deflects the preceding video frame forward by the third preset time duration to determine the first timestamp of the current video frame.

[0084] For example, the server reads the timestamp of video frame 300. After detection, there is no video frame indicating that the main vehicle passed through the key area earliest before video frame 300. Therefore, the first timestamp of video frame 300 is obtained by pushing the timestamp of video frame 300 forward 10 seconds; the server reads the timestamp of video frame 2000. After detection, the video frame indicating that the main vehicle passed through the key area earliest before video frame 2000 is video frame 1600. The first timestamp of video frame 2000 is obtained by pushing the timestamp of video frame 1600 forward 10 seconds.

[0085] 207. Obtain first and second vehicle information based on the first and second timestamps of each video frame.

[0086] The server obtains the main vehicle information in the first timestamp and the second timestamp corresponding to each video frame, determines the main vehicle information in the first timestamp as the first main vehicle information, and determines the main vehicle information in the second timestamp as the second main vehicle information.

[0087] 208. Determine whether the vehicle deviates from the original route based on the first vehicle information and the second vehicle information corresponding to each video frame. If so, remove the candidate key area scene where the corresponding video frame is located.

[0088] The server obtains the first orientation angle of the main vehicle in the first main vehicle information corresponding to each video frame and the second orientation angle of the main vehicle in the second main vehicle information, and calculates the angle between the first main vehicle orientation angle and the second main vehicle orientation angle corresponding to each video frame. The angle calculation method is the same as the orientation angle calculation method stated in step 203, and will not be repeated here. It is determined whether there is a video frame in each candidate key area scene with an angle within a preset angle range. If so, it is determined that the main vehicle deviates from the original route. The main vehicle deviating from the original route means that the main vehicle has turned around, turned left, or turned right. The candidate key area scenes where the main vehicle deviates from the original route are removed, and the scene start time and scene end time of each remaining candidate key area scene are recorded.

[0089] For example, the server obtains the first orientation angle of the main vehicle in the first main vehicle information and the second orientation angle of the main vehicle in the second main vehicle information, calculates the included angle of the first main vehicle orientation angle and the second main vehicle orientation angle, and when there is a video frame with an angle of 45°≤included angle≤160° in the candidate key area scene, it is considered that the main vehicle is turning left or turning right. When there is an angle of 160°<included angle≤180 in the candidate key area scene, it is considered that the main vehicle is turning around, and the scenes indicating that the main vehicle has turned around, turned left, and turned right are removed.

[0090] 209. Score the difficulty coefficient of the remaining candidate key area scenes.

[0091] The server obtains the minimum distance recorded in the remaining candidate key area scenes, and determines the difficulty coefficient of the scene based on the numerical value of the minimum distance. The smaller the minimum distance value, the higher the difficulty coefficient.

[0092] In an embodiment of the present invention, video frames of a main vehicle passing through a key area are screened out according to preset conditions to obtain an initial scene, non-motorized objects that pose a collision risk with the main vehicle are identified, and the non-motorized objects closest to the main vehicle are recorded to obtain candidate scenes. This is used to evaluate the difficulty coefficient of the scene, which helps to classify the difficulty of the scene. Other branch scenes in the candidate scenes are removed to obtain an interactive scene in which the main vehicle encounters a non-motorized object crossing the road while driving forward, thereby improving screening efficiency and accuracy.

[0093] The above describes the automatic driving scene screening method in the embodiment of the present invention. The following describes the automatic driving scene screening device in the embodiment of the present invention. Figure 5 In one embodiment of the present invention, an apparatus for screening autonomous driving scenes includes:

[0094] The acquisition module 501 is used to collect drive test data and extract the main vehicle information, key area information, and non-motorized object information from each video frame of the drive test data. The key area corresponding to the key area information is the area where the non-motorized object can move from one side of the road to the opposite side.

[0095] A screening module 502 is configured to screen video frames in which the main vehicle passes through a key area in the drive test data based on the main vehicle information and key area information in each video frame, and obtain at least one initial key area scene, where the initial key area scene is a sequence of video frames;

[0096] An identification module 503 is configured to identify non-motorized objects that pose a collision risk with the host vehicle in each frame of at least one initial key area scene, determine the identified non-motorized objects as candidate objects, and obtain at least one candidate object and corresponding at least one candidate object information;

[0097] a determination module 504 for determining a distance between the host vehicle and at least one candidate object based on the host vehicle information and at least one candidate object information, recording the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, and obtaining at least one candidate key area scene;

[0098] The removal module 505 is configured to remove a scene in which the host vehicle deviates from the original route from at least one candidate key area scene.

[0099] In an embodiment of the present invention, key interaction scenarios in which a vehicle passes through a key area are selected from road test data based on preset conditions, thereby improving screening efficiency. At the same time, identifying the distance between the vehicle and the nearest non-motorized object in the key interaction scenario helps to classify the difficulty of the scenario. Unnecessary subdivisions in the key interaction scenario are then removed to obtain interaction scenarios that meet the requirements, thereby improving screening accuracy.

[0100] See also Figure 6Another embodiment of the autonomous driving scene screening device in the embodiment of the present invention includes:

[0101] The acquisition module 501 is used to collect drive test data and extract the main vehicle information, key area information, and non-motorized object information from each video frame of the drive test data. The key area corresponding to the key area information is the area where the non-motorized object can move from one side of the road to the opposite side.

[0102] A screening module 502 is configured to screen video frames in which the main vehicle passes through a key area in the drive test data based on the main vehicle information and key area information in each video frame, and obtain at least one initial key area scene, where the initial key area scene is a sequence of video frames;

[0103] An identification module 503 is configured to identify non-motorized objects that pose a collision risk with the host vehicle in each frame of at least one initial key area scene, determine the identified non-motorized objects as candidate objects, and obtain at least one candidate object and corresponding at least one candidate object information;

[0104] a determination module 504 for determining a distance between the host vehicle and at least one candidate object based on the host vehicle information and at least one candidate object information, recording the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, and obtaining at least one candidate key area scene;

[0105] The removal module 505 is configured to remove a scene in which the host vehicle deviates from the original route from at least one candidate key area scene.

[0106] Optionally, the screening module 502 is specifically configured to:

[0107] Read the timestamp of each video frame in the drive test data, and use the timestamp of each video frame as a reference to push forward a first preset time length and push backward a second preset time length to obtain a preset time period corresponding to each video frame; obtain the main vehicle coordinates and key area range in each video frame based on the main vehicle information and key area information in each video frame; determine whether there is a video frame in each preset time period whose main vehicle coordinates are within the key area; if so, retain the video frame corresponding to the preset time period; if not, remove the video frame corresponding to the preset time period; divide the video frames retained in the drive test data based on the continuity of the timestamps to obtain at least one video frame sequence, each video frame sequence being an initial key area scene, and obtain at least one initial key area scene.

[0108] Optionally, the identification module 503 is specifically configured to:

[0109] Identify non-motorized objects within a preset distance range around the main vehicle in each frame of at least one initial key area scene, where the preset distance includes a preset lateral distance and a preset longitudinal distance; obtain the travel speed of each identified non-motorized object, and calculate the orientation angle between each identified non-motorized object and the main vehicle in the same video frame; when the travel speed is greater than the preset speed and the orientation angle is within the preset angle range, determine the corresponding non-motorized object as a candidate object, and obtain at least one candidate object; and obtain at least one candidate object information corresponding to the at least one candidate object.

[0110] Optionally, the determining module 504 includes:

[0111] The first reading unit 5041 is configured to read at least one candidate video frame corresponding to at least one candidate object, and obtain the contour information of the main vehicle and the coordinate points of the at least one candidate object in the at least one candidate video frame based on the main vehicle information and the at least one candidate object information;

[0112] a calculation unit 5042, configured to calculate the distance between each candidate object and the main vehicle in the same video frame based on the contour information of the main vehicle in the at least one candidate video frame and the coordinate points of the at least one candidate object;

[0113] The recording unit 5043 is configured to record the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, and obtain at least one candidate key area scene.

[0114] Optionally, the calculation unit 5042 is specifically configured to:

[0115] Based on the contour information of the main vehicle in at least one candidate video frame, the point on the contour of the main vehicle in each candidate video frame that is closest to the candidate object is scanned to obtain at least one candidate point, and the at least one candidate point has a one-to-one correspondence with the at least one candidate object; the distance between the at least one candidate point and the coordinate point of the at least one candidate object is calculated to obtain at least one contour distance, and the at least one contour distance has a one-to-one correspondence with the at least one candidate object.

[0116] Optionally, the removal module 505 includes:

[0117] The second reading unit 5051 is configured to read the timestamp of each video frame in at least one candidate key area scene, and to use the timestamp of each video frame as a reference, to push forward a third preset time length to obtain a first timestamp of each video frame, and to push backward a fourth preset time length to obtain a second timestamp of each video frame;

[0118] A determining unit 5052 is configured to obtain the main vehicle information in the first timestamp and the second timestamp corresponding to each video frame, determine the main vehicle information in the first timestamp as the first main vehicle information, and determine the main vehicle information in the second timestamp as the second main vehicle information;

[0119] The removal unit 5053 is configured to determine whether the main vehicle deviates from the original route based on the first main vehicle information and the second main vehicle information corresponding to each video frame, and if so, remove the candidate key area scene where the corresponding video frame is located.

[0120] Optionally, the removal unit 5053 is specifically configured to:

[0121] Obtain a first main vehicle heading angle from the first main vehicle information and a second main vehicle heading angle from the second main vehicle information; calculate the included angle between the first main vehicle heading angle and the second main vehicle heading angle; determine whether the included angle is within a preset angle range. If so, determine that the main vehicle has deviated from the original route, and remove the candidate key area scene where the video frame corresponding to the included angle is located.

[0122] In an embodiment of the present invention, video frames of a main vehicle passing through a key area are screened out according to preset conditions to obtain an initial scene, non-motorized objects that pose a collision risk with the main vehicle are identified, and the non-motorized objects closest to the main vehicle are recorded to obtain candidate scenes. This is used to evaluate the difficulty coefficient of the scene, which helps to classify the difficulty of the scene. Other branch scenes in the candidate scenes are removed to obtain an interactive scene in which the main vehicle encounters a non-motorized object crossing the road while driving forward, thereby improving screening efficiency and accuracy.

[0123] above Figure 5 and Figure 6 The autonomous driving scene screening device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The autonomous driving scene screening device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0124] Figure 7 Schematic diagram of the structure of an autonomous driving scene screening device provided by an embodiment of the present invention. The autonomous driving scene screening device 700 may vary significantly due to different configurations or performance, and may include one or more processors (central processing units, CPUs) 710 (for example, one or more processors) and memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. The memory 720 and storage medium 730 may be either transient or persistent storage. The program stored in the storage medium 730 may include one or more modules (not shown), each of which may include a series of instruction operations in the autonomous driving scene screening device 700. Furthermore, the processor 710 may be configured to communicate with the storage medium 730 to execute the series of instruction operations in the storage medium 730 on the autonomous driving scene screening device 700.

[0125] The autonomous driving scenario screening device 700 may further include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 760, and / or one or more operating systems 731, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that Figure 7 The structure of the autonomous driving scene screening device shown does not constitute a limitation of the autonomous driving scene screening device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0126] The present invention also provides an autonomous driving scenario screening device, wherein the computer device includes a memory and a processor, the memory storing computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the steps of the autonomous driving scenario screening method described in each of the above embodiments. The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the autonomous driving scenario screening method.

[0127] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program code.

[0129] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for screening autonomous driving scenarios, characterized in that: The autonomous driving scenario screening method includes: Collecting drive test data, extracting vehicle information, key area information, and non-motorized object information from each video frame of the drive test data, wherein the key area corresponding to the key area information is an area where a non-motorized object can move from one side of the road to the opposite side; Filtering video frames of the main vehicle passing through the key area in the drive test data based on the main vehicle information and key area information in each video frame to obtain at least one initial key area scene, where the initial key area scene is a video frame sequence; Identifying non-motorized objects that pose a collision risk with the host vehicle in each frame of the at least one initial key area scene, determining the identified non-motorized objects as candidate objects, and obtaining at least one candidate object and corresponding at least one candidate object information; determining a distance between the at least one candidate object and the host vehicle based on the host vehicle information and the at least one candidate object information, recording a minimum distance in each video frame and the candidate object information corresponding to the minimum distance, and obtaining at least one candidate key area scene; A scene in which the main vehicle deviates from the original route is removed from the at least one candidate key area scene, wherein the scene in which the main vehicle deviates from the original route is a scene in which the main vehicle makes a U-turn, turns left, or turns right near the key area.

2. The autonomous driving scenario screening method according to claim 1, characterized in that: The step of filtering out video frames in the drive test data in which the main vehicle passes through the key area based on the main vehicle information and the key area information in each video frame to obtain at least one initial key area scene includes: Reading the timestamp of each video frame in the drive test data, and using the timestamp of each video frame as a reference, moving forward a first preset time length and moving backward a second preset time length to obtain a preset time period corresponding to each video frame; Obtain the main vehicle coordinates and key area range in each video frame according to the main vehicle information and key area information in each video frame; Determine whether there are video frames in each preset time period whose coordinates of the main vehicle are within the key area. If so, retain the video frames corresponding to the preset time period; if not, remove the video frames corresponding to the preset time period. Based on the continuity of timestamps, the video frames retained in the drive test data are divided to obtain at least one video frame sequence, each video frame sequence being an initial key area scene, and at least one initial key area scene is obtained.

3. The autonomous driving scenario screening method according to claim 1, characterized in that: The step of identifying a non-motorized object that has a collision risk with the host vehicle in each frame of the at least one initial key area scene, determining the identified non-motorized object as a candidate object, and obtaining at least one candidate object and corresponding at least one candidate object information includes: Identifying non-motorized objects within a preset distance around the host vehicle in each frame of the at least one initial key area scene, wherein the preset distance includes a preset lateral distance and a preset longitudinal distance; Obtain the speed of each identified non-motorized object and calculate the angle between each identified non-motorized object and the main vehicle in the same video frame; When the traveling speed is greater than a preset speed and the orientation angle is within a preset angle range, the corresponding non-motorized object is determined as a candidate object to obtain at least one candidate object; At least one candidate object information corresponding to the at least one candidate object is obtained.

4. The autonomous driving scenario screening method according to claim 1, characterized in that: The step of determining the distance between the at least one candidate object and the host vehicle based on the host vehicle information and the at least one candidate object information, recording the minimum distance in each video frame and the candidate object information corresponding to the minimum distance, and obtaining at least one candidate key area scene includes: Reading at least one candidate video frame corresponding to the at least one candidate object, and obtaining, based on the main vehicle information and the at least one candidate object information, contour information of the main vehicle and coordinate points of the at least one candidate object in the at least one candidate video frame; Calculating the distance between each candidate object and the main vehicle in the same video frame based on the contour information of the main vehicle in the at least one candidate video frame and the coordinate points of the at least one candidate object; The minimum distance in each video frame and the candidate object information corresponding to the minimum distance are recorded to obtain at least one candidate key area scene.

5. The method for screening autonomous driving scenarios according to claim 4, wherein: The calculating, based on the contour information of the main vehicle in the at least one candidate video frame and the coordinate point of the at least one candidate object, the distance between each candidate object and the main vehicle in the same video frame includes: Scanning a point on the main vehicle outline in each candidate video frame that is closest to the candidate object based on the outline information of the main vehicle in the at least one candidate video frame to obtain at least one candidate point, where the at least one candidate point corresponds one-to-one to the at least one candidate object; A distance between the at least one candidate point and a coordinate point of the at least one candidate object is calculated to obtain at least one contour distance, where the at least one contour distance corresponds one-to-one to the at least one candidate object.

6. The autonomous driving scenario screening method according to claim 1, characterized in that: The removing of the scene in which the host vehicle deviates from the original route from the at least one candidate key area scene includes: Reading the timestamp of each video frame in the at least one candidate key area scene, and using the timestamp of each video frame as a reference, pushing forward a third preset time length to obtain a first timestamp of each video frame, and pushing backward a fourth preset time length to obtain a second timestamp of each video frame; Obtaining the main vehicle information in the first timestamp and the second timestamp corresponding to each video frame, determining the main vehicle information in the first timestamp as the first main vehicle information, and determining the main vehicle information in the second timestamp as the second main vehicle information; Whether the main vehicle deviates from the original route is determined based on the first main vehicle information and the second main vehicle information corresponding to each video frame. If so, the candidate key area scene where the corresponding video frame is located is removed.

7. The method for screening autonomous driving scenarios according to claim 6, wherein: The determining whether the main vehicle deviates from the original route based on the first main vehicle information and the second main vehicle information corresponding to each video frame, and if so, removing the candidate key area scene where the corresponding video frame is located, includes: Obtaining a first orientation angle of the main vehicle in the first main vehicle information and a second orientation angle of the main vehicle in the second main vehicle information; Calculating the included angle between the first main vehicle heading angle and the second main vehicle heading angle; It is determined whether the angle is within a preset angle range. If so, it is determined that the main vehicle has deviated from the original route, and the candidate key area scene where the video frame corresponding to the angle is located is removed.

8. An autonomous driving scene screening device, characterized in that: The automatic driving scene screening device includes: An acquisition module is configured to acquire drive test data and extract information about the vehicle, key areas, and non-motorized objects from each video frame of the drive test data, wherein the key areas corresponding to the key area information are areas where non-motorized objects can move from one side of the road to the opposite side; a screening module, configured to screen out video frames in the drive test data where the vehicle passes through the key area based on the vehicle information and key area information in each video frame, and obtain at least one initial key area scene, where the initial key area scene is a sequence of video frames; an identification module, configured to identify non-motorized objects that pose a collision risk with the host vehicle in each frame of the at least one initial key area scene, determine the identified non-motorized objects as candidate objects, and obtain at least one candidate object and corresponding at least one candidate object information; a determination module, configured to determine a distance between the at least one candidate object and the host vehicle based on the host vehicle information and the at least one candidate object information, record a minimum distance in each video frame and the candidate object information corresponding to the minimum distance, and obtain at least one candidate key area scene; The removal module is configured to remove, from the at least one candidate key area scene, a scene in which the main vehicle deviates from the original route, wherein the scene in which the main vehicle deviates from the original route is a scene in which the main vehicle makes a U-turn, turns left, or turns right near the key area.

9. An autonomous driving scene screening device, characterized in that: The autonomous driving scenario screening device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory so that the autonomous driving scene screening device executes the autonomous driving scene screening method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, the autonomous driving scenario screening method as described in any one of claims 1 to 7 is implemented.

Images