Driving area map construction method, system and equipment based on aerial view image

Abstract

The embodiment of the invention relates to the field of intelligent traffic, and provides a driving area map construction method, system and device based on an aerial view image, and a storage medium, and the method comprises the steps: obtaining a target aerial view image sequence corresponding to the traffic flow of a target road section; according to the target aerial view image sequence, obtaining a traffic participant trajectory data set of the target road section; screening and classifying the traffic participant trajectory data set to obtain a target motor vehicle trajectory data set corresponding to each route in the at least one route of the target road section; and according to the target motor vehicle track data set corresponding to each route, constructing a driving area map corresponding to each route in the at least one route of the target road section. The technical scheme provided by the embodiment of the invention is suitable for map construction of structured road sections with clear lane markings and good traffic order, is also suitable for map construction of unstructured road sections with fuzzy lane markings and even without lane markings, and is also suitable for map construction of unstructured road sections on which motor vehicles do not strictly run according to the lane markings; and the accuracy and the universality of the map construction of the driving area are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a method, system, device and storage medium for constructing a driving area map based on a bird's-eye view image. Background Art

[0002] Maps are an important component of natural driving datasets, especially in human driver behavior research and high-level autonomous driving tests. Maps provide the basic road information foundation for extracting vehicle behavior and reproducing scenes. Currently, map construction methods mainly include manually drawing standard format maps or using deep learning networks to identify road markings to build maps. Manual drawing or relying on deep learning is suitable for map construction of structured roads with clear lane markings and good traffic order, for example, for map construction of structured roads such as highways, expressways, and intersections.

[0003] However, the above methods are not suitable for mapping unstructured roads. Unstructured roads can include sections with blurred or even no lane markings (such as snow-covered roads, roads with severely worn lane markings, and country roads), and sections where vehicles do not strictly follow lane markings (such as urban roads with mixed pedestrian and vehicle traffic, and sections where lanes are occupied by illegally parked vehicles or obstacles). In other words, manual mapping or deep learning methods that rely on the limited information within the image cannot accurately build maps of unstructured roads. Summary of the Invention

[0004] The embodiments of the present invention provide a method, system, device and storage medium for constructing a driving area map based on a bird's-eye view image, aiming to improve the accuracy of constructing the driving area map.

[0005] In a first aspect, an embodiment of the present invention provides a method for constructing a driving area map based on a bird's-eye view image, comprising:

[0006] Obtain a target bird's-eye view image sequence corresponding to the traffic flow of the target road section;

[0007] Acquiring a traffic participant trajectory dataset of the target road section according to the target bird's-eye view image sequence;

[0008] Screening and classifying the traffic participant trajectory dataset to obtain a target motor vehicle trajectory dataset corresponding to each of the at least one route of the target road section;

[0009] A driving area map corresponding to each of the at least one route of the target road section is constructed based on the target motor vehicle trajectory dataset corresponding to each of the routes.

[0010] In a second aspect, an embodiment of the present invention further provides a computer device, comprising a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the driving area map construction method as described in the first aspect is implemented.

[0011] In a third aspect, an embodiment of the present invention further provides a system for constructing a driving area map based on a bird's-eye view image, comprising: a height maintenance device, an image acquisition device, and a computer device, wherein:

[0012] The height maintaining device is deployed on the target road section and is used to carry the image acquisition device, so that the image acquisition device can acquire a plurality of original bird's-eye view image sequences of the traffic flow of the target road section from a bird's-eye view perspective;

[0013] The image acquisition device is mounted on the altitude maintaining device and is used to acquire a plurality of original bird's-eye view image sequences of the traffic flow of the target road section from a bird's-eye view perspective;

[0014] The computer device is used to obtain the multiple original bird's-eye view image sequences and implement the driving area map construction method as described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present invention further provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the driving area map construction method as described in the first aspect.

[0016] An embodiment of the present invention provides a method, system, device and storage medium for constructing a driving area map based on a bird's-eye view image. The embodiment of the present invention uses a target motor vehicle trajectory dataset obtained from a target bird's-eye view image sequence corresponding to the traffic flow of the target road section to construct a driving area map of the target road section. The method does not rely on the lane markings of the target road section, nor does it limit the motor vehicle to strictly follow the lane markings in the target road section. Therefore, regardless of whether the target road section is a structured road section with clear lane markings and good traffic order, or an unstructured road section with blurred lane markings or even no lane markings, or an unstructured road section where the motor vehicle does not strictly follow the lane markings, the target motor vehicle trajectory dataset obtained from the target bird's-eye view image sequence corresponding to the traffic flow of the target road section can be used to accurately construct a driving area map of the target road section, thereby effectively improving the accuracy and universality of driving area map construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic flow chart of a method for constructing a driving area map based on a bird's-eye view image provided by an embodiment of the present invention;

[0019] Figure 2 is a schematic diagram of a bird's-eye view image of a target road section acquired by a first unmanned aerial vehicle in an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of a bird's-eye view image of a target road section acquired by a second unmanned aerial vehicle in an embodiment of the present invention;

[0021] Figure 4 is an example diagram of parameters included in the static data of traffic participants in an embodiment of the present invention;

[0022] Figure 5 yes Figure 1 A schematic flow chart of the sub-steps of the driving area map construction method in FIG.

[0023] Figure 6 1 is a schematic diagram of a process for correcting the position and shape of a detection frame to be corrected according to an embodiment of the present invention;

[0024] Figure 7 yes Figure 1 A schematic flow chart of the sub-steps of the driving area map construction method in FIG.

[0025] Figure 8 is a schematic diagram of routes of multiple target clusters corresponding to a target road segment in an embodiment of the present invention;

[0026] Figure 9 is another schematic diagram of routes of multiple target clusters corresponding to a target road segment in an embodiment of the present invention;

[0027] Figure 10 This is a schematic diagram of a process for constructing a driving area map based on a target motor vehicle trajectory dataset in an embodiment of the present invention;

[0028] Figure 11 is a schematic diagram of a driving area map in an embodiment of the present invention;

[0029] Figure 12 is another schematic diagram of a driving area map in an embodiment of the present invention;

[0030] Figure 13 1 is a schematic diagram comparing a driving area map and a standard lane map in an embodiment of the present invention;

[0031] Figure 14 This is a schematic block diagram of the structure of a computer device provided by an embodiment of the present invention;

[0032] Figure 15 The present invention provides a schematic structural diagram of a system for constructing a driving area map based on a bird's-eye view image. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0034] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0035] It should be understood that the terms used in this specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0036] Maps are an important component of natural driving datasets, especially in human driver behavior research and high-level autonomous driving tests. Maps provide the basic road information foundation for extracting vehicle behavior and reproducing scenes. Currently, map construction methods mainly include manually drawing standard format maps or using deep learning networks to identify road markings to build maps. Manual drawing or relying on deep learning is suitable for map construction of structured roads with clear lane markings and good traffic order, for example, for map construction of structured roads such as highways, expressways, and intersections.

[0037] However, the above methods are not suitable for mapping unstructured roads. Unstructured roads can include sections with blurred or even no lane markings (such as snow-covered roads, roads with severely worn lane markings, and country roads), and sections where vehicles do not strictly follow lane markings (such as urban roads with mixed pedestrian and vehicle traffic, and sections where lanes are occupied by illegally parked vehicles or obstacles). In other words, manual mapping or deep learning methods that rely on the limited information within the image cannot accurately build maps of unstructured roads.

[0038] To solve the above problems, a method, system, device and storage medium for constructing a driving area map based on bird's-eye view images are provided. The embodiments of the present invention use a target motor vehicle trajectory dataset obtained from a target bird's-eye view image sequence corresponding to the traffic flow of the target road section to construct a driving area map of the target road section. The method does not rely on the lane markings of the target road section, nor does it require motor vehicles to strictly follow the lane markings on the target road section. Therefore, regardless of whether the target road section is a structured road section with clear lane markings and good traffic order, or an unstructured road section with blurred lane markings or even no lane markings, or an unstructured road section where motor vehicles do not strictly follow the lane markings, the target motor vehicle trajectory dataset obtained from the target bird's-eye view image sequence corresponding to the traffic flow of the target road section can be used to accurately construct a driving area map of the target road section, effectively improving the accuracy and universality of driving area map construction.

[0039] In an embodiment of the present invention, a method for constructing a driving area map based on a bird's-eye view image can be applied to a computer device, which may include a terminal device or a server. The terminal device may include a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device, etc. The server may be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0040] The following embodiments of the present invention are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0041] See also Figure 1 , Figure 1 The present invention provides a flowchart of a method for constructing a driving area map based on a bird's-eye view image.

[0042] like Figure 1 As shown, the driving area map construction method includes steps S101 to S104.

[0043] Step S101: Acquire a target bird's-eye view image sequence corresponding to the traffic flow of a target road section.

[0044] In this embodiment, the target road section may include a structured road section with clear lane markings and good traffic order, for example, a highway, expressway, or intersection. The target road section may also include an unstructured road section with blurred or even no lane markings, such as a snow-covered road section, a road with severely worn lane markings, or a country road. The target road section may also include a road section where motor vehicles do not strictly follow lane markings, such as an urban road section with mixed pedestrian and vehicle traffic, or a road section where the lane area is occupied by illegally parked vehicles or obstacles.

[0045] In some embodiments, the target bird's-eye view image sequence is obtained by capturing the traffic flow of the target road section from a bird's-eye view using an image acquisition device mounted on an altitude maintenance device deployed on the target road section. In response to the angle between the shooting optical axis of the image acquisition device and the road surface where the target road section is located being within a preset angle range, the image acquisition perspective of the image acquisition device is determined to be a bird's-eye view. The preset angle range can be set by the user or by default. For example, the preset angle range includes [60°, 90°]. The altitude maintenance device can include an unmanned aerial vehicle, a bracket, or a street lamp, and the image acquisition device can include a digital camera, a single-lens reflex camera, an infrared camera, or a depth camera.

[0046] In some embodiments, obtaining a target bird's-eye view image sequence corresponding to the traffic flow of a target road section may include: obtaining an original bird's-eye view image sequence obtained by an image acquisition device capturing the traffic flow of the target road section from a bird's-eye view perspective; and performing image stabilization processing on the original bird's-eye view image sequence to obtain the target bird's-eye view image sequence. This embodiment, by performing image stabilization processing on the original bird's-eye view image sequence captured by the image acquisition device, eliminates video jitter caused by vibration of the image acquisition device itself and environmental influences, thereby improving the accuracy of the subsequent acquisition of the traffic participant trajectory dataset.

[0047] In some embodiments, obtaining a target bird's-eye view image sequence corresponding to the traffic flow of a target road section may include: obtaining multiple original bird's-eye view image sequences obtained by an image acquisition device capturing the traffic flow of the target road section from a bird's-eye view perspective; performing image stabilization processing on each of the multiple original bird's-eye view image sequences, and aligning the multiple original bird's-eye view image sequences after image stabilization processing to the same image coordinate system to obtain multiple target bird's-eye view image sequences. This embodiment eliminates video jitter caused by the vibration of the image acquisition device itself and environmental influences by performing image stabilization processing on the original bird's-eye view image sequences, thereby improving the accuracy of subsequent acquisition of traffic participant trajectory data sets. By aligning the multiple original bird's-eye view image sequences after image stabilization processing to the same image coordinate system, it is possible to eliminate image field deviations caused by position deviations of multiple task acquisitions, so that all trajectory data subsequently acquired from the same point can share the same coordinate system, and only one bird's-eye view image at a point needs to be mapped to meet the needs of all trajectory data for map use, thereby significantly improving the processing efficiency of map construction.

[0048] In some embodiments, multiple original bird's-eye view image sequences may include original bird's-eye view image sequences collected by the same image acquisition device in different time periods and / or original bird's-eye view image sequences collected by different image acquisition devices in the same or different time periods. For example, during the morning and evening rush hours, the first unmanned aerial vehicle is controlled to hover above the target road section at a set altitude (for example, 80 meters), and the image acquisition device carried on it is controlled to collect data on the traffic flow from a bird's-eye view perspective (the angle between the shooting optical axis of the image acquisition device and the road surface where the target road section is located is 90°), and the five original bird's-eye view image sequences shown in Table 1 are obtained, including the original bird's-eye view image sequence collected by the image acquisition device carried by the first unmanned aerial vehicle from 8:50 to 9:50 on July 19, 2024, ... The original bird's-eye view image sequence is collected by the device between 3:00 PM and 4:30 PM on July 19, 2024; the original bird's-eye view image sequence is collected by the image acquisition device on the first unmanned aerial vehicle between 10:20 PM and 10:50 PM on July 19, 2024; the original bird's-eye view image sequence is collected by the image acquisition device on the first unmanned aerial vehicle between 9:30 AM and 10:00 AM on September 19, 2024; and the original bird's-eye view image sequence is collected by the image acquisition device on the first unmanned aerial vehicle between 3:00 PM and 4:40 PM on September 19, 2024. Stabilization is performed on each of these five original bird's-eye view image sequences, and the five stabilized original bird's-eye view image sequences are aligned to the same coordinate system to obtain five target bird's-eye view image sequences.

[0049] Table 1

[0050]

[0051]

[0052] For another example, during the morning and evening rush hours, the first unmanned aerial vehicle is controlled to hover at a set altitude (for example, 80 meters) above the first position of the target road section, and the second unmanned aerial vehicle is controlled to hover at a set altitude (for example, 80 meters) above the second position of the target road section. Then, the image acquisition device carried by the first unmanned aerial vehicle is controlled to collect traffic flow data from a bird's-eye view (the angle between the shooting optical axis of the image acquisition device and the road surface where the target road section is located is 90°). At the same time, the image acquisition device carried by the second unmanned aerial vehicle is controlled to collect traffic flow data from a bird's-eye view (the angle between the shooting optical axis of the image acquisition device and the road surface where the target road section is located is 90°). Five original bird's-eye view image sequences are obtained as shown in Table 1, and a total of 10 original bird's-eye view image sequences are obtained. The bird's-eye view image sequences collected by the image acquisition device carried by the first unmanned aerial vehicle above the first position of the target road section include: Figure 2 The bird's-eye view image sequence shown in FIG. 1 includes the following: Figure 3 The bird's-eye view image shown in FIG. 10 is a stabilization process performed on each of the 10 original bird's-eye view image sequences, and the 10 stabilized original bird's-eye view image sequences are aligned to the same coordinate system to obtain 10 target bird's-eye view image sequences.

[0053] The above 10 original bird's-eye view image sequences include the original bird's-eye view image sequences collected by the image acquisition device carried by the first unmanned aerial vehicle and the image acquisition device carried by the second unmanned aerial vehicle respectively from 8:50 to 9:50 on July 19, 2024, the original bird's-eye view image sequences collected by the image acquisition device carried by the first unmanned aerial vehicle and the image acquisition device carried by the second unmanned aerial vehicle respectively from 15:00 to 16:30 on July 19, 2024, and the original bird's-eye view image sequences collected by the image acquisition device carried by the first unmanned aerial vehicle and the image acquisition device carried by the second unmanned aerial vehicle respectively from 15:00 to 16:30 on July 19, 2024. The original bird's-eye image sequence was collected by the image acquisition device carried by the first unmanned aerial vehicle from 22:20 to 22:50 on July 19, 2024, the original bird's-eye image sequence was collected by the image acquisition device carried by the first unmanned aerial vehicle and the image acquisition device carried by the second unmanned aerial vehicle from 9:30 to 10:00 on September 19, 2024, and the original bird's-eye image sequence was collected by the image acquisition device carried by the first unmanned aerial vehicle and the image acquisition device carried by the second unmanned aerial vehicle from 15:00 to 16:40 on September 19, 2024.

[0054] When an unmanned aerial vehicle (UAV) collects bird's-eye view image sequences, the image sequences can experience jitter due to environmental factors such as UAV vibration and strong winds. This can cause rotation and translation in the preceding and following image frames, leading to positional errors in the subsequent trajectory acquisition based on the image coordinate system. Therefore, by performing image stabilization on each of the 10 original bird's-eye view image sequences, this jitter caused by environmental factors such as UAV vibration and strong winds is eliminated, thereby improving the accuracy of the subsequent traffic participant trajectory dataset. Due to the limitations of the UAV's battery life, during the execution of the bird's-eye view image sequence collection task, the UAV needs to be controlled to land and replace the battery. After replacing the battery, it returns to the sky above the target road section to continue collecting the bird's-eye view image sequence. Under the influence of reciprocating takeoff and landing, the points collected by the UAV each time cannot be consistent, resulting in field of view deviations between different original bird's-eye view image sequences. The field of view deviations between different original bird's-eye view image sequences are mainly caused by changes in collection height, direction and position, as well as perspective deviations of surrounding buildings. Therefore, by aligning the above 10 original bird's-eye view image sequences to a unified image coordinate system, the field of view deviations between different original bird's-eye view image sequences can be eliminated, thereby significantly improving the processing efficiency of subsequent map construction and trajectory alignment.

[0055] Step S102: Acquire a data set of traffic participant trajectories of a target road section based on the target bird's-eye view image sequence.

[0056] In this embodiment, the traffic participant trajectory dataset includes multiple traffic participant trajectory data. Traffic participant trajectory data refers to the trajectory data of traffic participants. The traffic participant trajectory dataset may include the trajectory data of each traffic participant in the target bird's-eye view image sequence. Traffic participants may include cars, trucks, buses, tricycles (including electric tricycles or gasoline tricycles), motorcycles (including electric motorcycles or gasoline motorcycles), bicycles, or pedestrians.

[0057] In some embodiments, the traffic participant trajectory data includes static data and dynamic data of the traffic participant. The dynamic data of the traffic participant may include the speed curve of the traffic participant during the acquisition time period of the target bird's-eye view image sequence. The static data of the traffic participant includes the position information, length, width and orientation angle of the traffic participant at the acquisition time of each bird's-eye view image in the target bird's-eye view image sequence. The position information includes the position coordinates of the center point of the traffic participant and the position coordinates of the four corner points of the directional detection frame of the traffic participant. For example, Figure 4 As shown, the static data of the traffic participant 110 includes the length L of the acquisition time of the i-th bird's-eye view image in the target bird's-eye view image sequence, the width W, the orientation angle α of the traffic participant 110, the position coordinates (x i,y i ), the position coordinates (x ) of the four corner points of the directional detection frame 120 of the traffic participant 110 1i, y 1i ), (x 2i, y 2i ), (x 3i, y 3i ), (x 4i, y 4i ). i is an integer greater than or equal to 1.

[0058] In some embodiments, as Figure 5 As shown, step S102 includes sub-steps S1021 to S1025.

[0059] Sub-step S1021: calling a preset traffic participant detection model to perform traffic participant detection on each bird's-eye view image in the target bird's-eye view image sequence to obtain a traffic participant detection result for each bird's-eye view image.

[0060] In this embodiment, the traffic participant detection results of the bird's-eye view image may include the directional detection box, orientation angle, and traffic participant category of each traffic participant in the bird's-eye view image. Traffic participant categories may include pedestrians, bicycles, electric motorcycles, electric tricycles, cars, trucks, and buses. It should be noted that traffic participant categories may also include cars, trucks, buses, tricycles (including electric tricycles or gasoline tricycles), motorcycles (including electric motorcycles or gasoline motorcycles), bicycles, or pedestrians.

[0061] In some embodiments, the preset traffic participant detection model is obtained by iteratively training the target detection model in advance based on a training sample data set, where the sample data in the training sample set includes a sample image and an annotated directional detection box, an annotated orientation angle, and an annotated traffic participant category. The target detection model may include a YOLO model, an SSD (Single Shot MultiBox Detector) model, or a Faster R-CNN model, and the YOLO model may include a YOLOv11 model. For example, the YOLOv11 model may include five different versions: YOLOv11n-obb, YOLOv11m-obb, YOLOv11l-obb, YOLOv11s-obb, and YOLOv11x-obb.

[0062] For example, the preset traffic participant detection model is obtained by iteratively training the YOLOv11m-obb model based on the training sample data set in advance to ensure the detection accuracy of the traffic participant detection model. For example, the process of iteratively training the target detection model based on the training sample data set may include: obtaining a training sample from the training sample data set as a target training sample; inputting the sample image in the target training sample into the YOLOv11m-obb model to obtain the predicted directional detection frame, the predicted orientation angle, and the predicted probability that the traffic participant in the sample image belongs to each traffic participant category; determining the intersection-over-union ratio between the predicted directional detection frame and the labeled directional detection frame in the target training sample, calculating the cross entropy loss value based on the labeled traffic participant category in the target training sample and the predicted probability that the traffic participant in the sample image belongs to each traffic participant category, and calculating the distribution focus loss value based on the predicted orientation angle and the labeled orientation angle in the target training sample; when the intersection-over-union ratio is less than a preset intersection-over-union ratio threshold , update the parameters of the YOLOv11m-obb model that affect the intersection-union ratio according to the intersection-union ratio, when the cross-entropy loss value is less than the first loss value threshold, update the parameters of the YOLOv11m-obb model that affect the cross-entropy loss value according to the cross-entropy loss value, when the distribution focus loss value is less than the second loss value threshold, update the parameters of the YOLOv11m-obb model that affect the distribution focus loss value according to the distribution focus loss value, and then return to execute the step of obtaining a training sample from the training sample dataset as the target training sample; when the intersection-union ratio is greater than or equal to the preset intersection-union ratio threshold, the cross-entropy loss value is greater than or equal to the first loss value threshold, and the distribution focus loss value is greater than or equal to the second loss value threshold, stop iterative training of the YOLOv11m-obb model to obtain a traffic participant detection model.

[0063] Sub-step S1022: Determine static data of each traffic participant in each bird's-eye view image based on the traffic participant detection result of each bird's-eye view image.

[0064] In this embodiment, the traffic participant detection results of the bird's-eye view image may include the position coordinates, orientation angle and traffic participant category of the four corner points of the directional detection frame of each traffic participant in the bird's-eye view image. For each traffic participant, the position coordinates of the center point of the traffic participant and the length and width of the traffic participant can be calculated based on the position coordinates of the four corner points of the directional detection frame of the traffic participant. The position coordinates of the center point of the traffic participant, the position coordinates of the four corner points of the directional detection frame of the traffic participant, the orientation angle of the traffic participant and the length and width of the traffic participant are used as static data of the traffic participant.

[0065] Sub-step S1023: Based on the static data of each traffic participant in each bird's-eye view image, multi-target tracking is performed on the traffic participants in each bird's-eye view image, so as to assign a tracking ID to each successfully tracked traffic participant.

[0066] In this embodiment, a multi-target tracking algorithm can be used to track the traffic participants in each bird's-eye view image based on the static data of each traffic participant in each bird's-eye view image, thereby assigning a tracking ID to each successfully tracked traffic participant in each bird's-eye view image. The multi-target tracking algorithm can include a Kalman filter-based multi-target tracking algorithm, a Hungarian algorithm, or a deep learning-based multi-target tracking algorithm. The deep learning-based multi-target tracking algorithm can include a ByteTrack algorithm or a TransTrack algorithm.

[0067] Sub-step S1024: According to the tracking ID of each traffic participant, the static data of each traffic participant in each bird's-eye view image is associated to obtain a static data sequence corresponding to each tracking ID.

[0068] In this embodiment, the static data of each traffic participant in each bird's-eye view image can be saved according to the format of Table 2. The meanings of the parameters in Table 2 are: the tracking id in the i-th frame bird's-eye view image is id i The position coordinates of the center point of the traffic participant are (x i, y i ), the length and width of the oriented detection box are (w i , h i ) and the rotation angle of the directional detection frame around the positive direction of the x-axis (the direction of the traffic participant) is r i .

[0069] Table 2

[0070] frame id x y width height rotation i <![CDATA[id i ]]> <![CDATA[x i ]]> <![CDATA[y i ]]> <![CDATA[w i ]]> <![CDATA[h i ]]> <![CDATA[r i ]]>

[0071] Sub-step S1025: Correct the static data in the static data sequence corresponding to each tracking ID to obtain a target static data sequence corresponding to each tracking ID, and aggregate each target static data sequence to obtain a traffic participant trajectory dataset of the target road section.

[0072] This embodiment corrects the static data in the static data sequence corresponding to each tracking ID to eliminate perspective errors, making the corrected target static data closer to reality and further improving the accuracy of the traffic participant trajectory dataset. Specifically, this embodiment uses an improved L-shape algorithm to correct the static data in the static data sequence corresponding to each tracking ID.

[0073] In some embodiments, the static data in the static data sequence corresponding to the tracking ID is corrected to obtain a target static data sequence corresponding to the tracking ID, including: obtaining the directional detection frame closest to the minimum perspective distortion area from the directional detection frames contained in each static data in the static data sequence as a reference detection frame, and determining the size of the reference detection frame as the correction reference size; determining the directional detection frames other than the reference detection frame in the directional detection frames contained in each static data in the static data sequence as detection frames to be corrected; for each detection frame to be corrected, determining the corner point closest to the reference detection frame among the four corner points of the detection frame to be corrected as the correction reference corner point; and correcting the position and shape of each detection frame to be corrected in the static data sequence based on the correction reference corner point and correction reference size of each detection frame to be corrected to obtain the target static data sequence. This embodiment can eliminate perspective errors by correcting the position and shape of each detection frame to be corrected in the static data sequence, so that the corrected target static data can be closer to the actual situation, further improving the accuracy of the traffic participant trajectory dataset.

[0074] For example, see Figure 6 , Figure 6 FIG. 1 is a schematic diagram of a process for correcting the position and shape of a detection frame to be corrected according to an embodiment of the present invention. Figure 6 The bird's-eye view image shown in a includes a detection frame 11 of a traffic participant to be corrected. Figure 6 The bird's-eye view image shown in b includes an L-shape correction reference 12 based on the correction reference size and the correction reference corner points of the detection frame to be corrected 11. The position and shape of the detection frame to be corrected 11 are corrected according to the L-shape correction reference 12, and the following can be obtained: Figure 6 c shows the corrected directional detection frame 13 , which is shorter than the detection frame 11 to be corrected.

[0075] In some embodiments, when there are multiple target bird's-eye image sequences, for each target bird's-eye image sequence, a preset traffic participant detection model is called to perform traffic participant detection on each bird's-eye image in the target bird's-eye image sequence to obtain traffic participant detection results for each bird's-eye image in each target bird's-eye image sequence; based on the traffic participant detection results for each bird's-eye image in each target bird's-eye image sequence, static data of each traffic participant in each bird's-eye image in each target bird's-eye image sequence is determined; based on the static data of each traffic participant in each bird's-eye image in each target bird's-eye image sequence, multi-target tracking is performed on each traffic participant in each bird's-eye image in each target bird's-eye image sequence to assign a tracking ID to each successfully tracked traffic participant; based on the tracking ID of each successfully tracked traffic participant, the static data of each traffic participant in each bird's-eye image in each target bird's-eye image sequence is associated to obtain a static data sequence corresponding to each tracking ID; the static data in the static data sequence corresponding to each tracking ID is corrected to obtain a target static data sequence corresponding to each tracking ID, and each target static data sequence is aggregated to obtain a traffic participant trajectory dataset for the target road section.

[0076] Step S103 : screening and classifying the traffic participant trajectory dataset to obtain a target motor vehicle trajectory dataset corresponding to each route of at least one route of the target road section.

[0077] This embodiment can eliminate invalid trajectory data and classify the traffic participant trajectory dataset by screening and classifying it to obtain the target vehicle trajectory dataset corresponding to each route, thereby ensuring the accuracy of the target vehicle trajectory dataset.

[0078] In some embodiments, as Figure 7 As shown, step S103 includes sub-steps S1031 to S1034.

[0079] Sub-step S1031 : Filtering motor vehicle trajectory data from the traffic participant trajectory dataset to obtain a first motor vehicle trajectory dataset.

[0080] This embodiment, by filtering out the motor vehicle trajectory data, can eliminate irregular pedestrian and non-motor vehicle trajectory data to ensure the validity and accuracy of the motor vehicle trajectory dataset used for clustering. The first motor vehicle trajectory dataset is a subset of the traffic participant trajectory dataset. Motor vehicle trajectory data refers to the trajectory data of traffic participants whose traffic participant category is motor vehicle, that is, the motor vehicle trajectory data is the trajectory data of motor vehicles.

[0081] In some embodiments, the first vehicle trajectory dataset may include trajectory data of a car, a truck, and a bus. Alternatively, the first vehicle trajectory dataset may include trajectory data of a car, a truck, a bus, and a motorcycle. For example, the traffic participant trajectory dataset may include trajectory data of a car, a truck, a bus, a tricycle, a motorcycle, a bicycle, and a pedestrian.

[0082] Sub-step S1032 : Filter out the motor vehicle trajectory data sets whose trajectory duration is greater than or equal to the target duration threshold from the first motor vehicle trajectory data set to obtain a second motor vehicle trajectory data set.

[0083] This embodiment filters out vehicle trajectory data with trajectory durations greater than or equal to a target duration threshold. This can eliminate vehicle trajectory data that has been stationary for a long time or has incomplete trajectories, thereby improving the accuracy of the vehicle trajectory dataset. Furthermore, it can reduce the amount of data used for clustering, thereby improving clustering efficiency and accuracy.

[0084] In some embodiments, the target duration threshold is pre-set by the user or determined based on the distribution of trajectory durations corresponding to each vehicle trajectory data set in the first vehicle trajectory dataset. For example, a box-shaped distribution plot of trajectory durations is generated based on the trajectory durations corresponding to each vehicle trajectory data set in the first vehicle trajectory dataset. The first and third quartiles of the trajectory durations are determined based on the box-shaped distribution plot. The interquartile range is obtained by subtracting the first quartile from the third quartile, and the target duration threshold is obtained by subtracting 1.5 times the interquartile range from the first quartile.

[0085] Sub-step S1033 : performing clustering processing on the second motor vehicle trajectory dataset to obtain clusters with the same number of target centroids, where the clusters are subsets of the second motor vehicle trajectory dataset.

[0086] In this embodiment, the second vehicle trajectory dataset can be clustered based on a preset clustering algorithm to obtain clusters containing target centroids. The clusters are subsets of the second vehicle trajectory dataset. The preset clustering algorithm can be user-configured and is not specifically limited in this embodiment. For example, the preset clustering algorithm includes a traditional K-means algorithm or a K-means++ algorithm.

[0087] In some embodiments, clustering the second motor vehicle trajectory data set to obtain a cluster with a target number of centroids may include: randomly selecting a motor vehicle trajectory data from the second motor vehicle trajectory data set as the cluster centroid; determining the minimum distance between each motor vehicle trajectory data in the second motor vehicle trajectory data set except the cluster centroid and the cluster centroid; determining the probability that each motor vehicle trajectory data in the second motor vehicle trajectory data set except the cluster centroid is a new cluster centroid based on the minimum distance between each motor vehicle trajectory data in the second motor vehicle trajectory data set except the cluster centroid and the cluster centroid, and using the motor vehicle trajectory data corresponding to the maximum probability as the new cluster centroid; returning to the step of determining the minimum distance between each motor vehicle trajectory data in the second motor vehicle trajectory data set except the cluster centroid and the cluster centroid until the number of determined cluster centroids reaches the target number; for all cluster centroids in the second motor vehicle trajectory data set except all cluster centroids, For each motor vehicle trajectory data other than the first one, determine the distance between the motor vehicle trajectory data and each cluster centroid, assign the motor vehicle trajectory data to the cluster centroid corresponding to the minimum distance, and obtain the initial cluster cluster with the target number of centroids; for each initial cluster cluster, determine the sum of the distances between each motor vehicle trajectory data in the initial cluster and the other motor vehicle trajectory data in the initial cluster; use the motor vehicle trajectory data corresponding to the minimum distance sum as the new cluster centroid of the initial cluster; determine the distance between the new cluster centroid of the initial cluster and the previous cluster centroid, and when the distance between the new cluster centroid of the initial cluster and the previous cluster centroid is greater than a preset distance threshold, return to the step of determining the sum of the distances between each motor vehicle trajectory data in the initial cluster and the other motor vehicle trajectory data in the initial cluster, until the distance between the new cluster centroid of the initial cluster and the previous cluster centroid is less than or equal to the preset distance threshold, and obtain the cluster cluster with the target number of centroids.

[0088] In some embodiments, clustering the second vehicle trajectory dataset to obtain clusters with a target number of centroids may include: normalizing each vehicle trajectory data in the second vehicle trajectory dataset to obtain a third vehicle trajectory dataset; and clustering the third vehicle trajectory dataset to obtain clusters with a target number of centroids, where the clusters are subsets of the third vehicle trajectory dataset. This embodiment further improves clustering efficiency and accuracy by normalizing each vehicle trajectory data in the vehicle trajectory dataset before performing clustering.

[0089] In some embodiments, normalizing each vehicle trajectory data in the second vehicle trajectory dataset to obtain the third vehicle trajectory dataset may include: unifying the trajectory length of each vehicle trajectory data in the second vehicle trajectory dataset to obtain a candidate vehicle trajectory dataset; and performing mean-variance normalization on each vehicle trajectory data in the candidate vehicle trajectory dataset to obtain the third vehicle trajectory dataset.

[0090] In some embodiments, unifying the trajectory length of each motor vehicle trajectory data in the second motor vehicle trajectory dataset to obtain the candidate motor vehicle trajectory dataset may include: determining a standard trajectory length based on the trajectory length of each motor vehicle trajectory data in the second motor vehicle trajectory dataset; for each motor vehicle trajectory data in the second motor vehicle trajectory dataset, in response to the trajectory length of the motor vehicle trajectory data being greater than the standard trajectory length, downsampling the motor vehicle trajectory data to shorten the trajectory length of the motor vehicle trajectory data so that the trajectory length of the motor vehicle trajectory data reaches the standard trajectory length; and in response to the trajectory length of the motor vehicle trajectory data being less than the standard trajectory length, performing linear interpolation processing on the motor vehicle trajectory data to lengthen the trajectory length of the motor vehicle trajectory data so that the trajectory length of the motor vehicle trajectory data reaches the standard trajectory length.

[0091] In some embodiments, determining the standard trajectory length based on the trajectory length of each vehicle trajectory data in the second vehicle trajectory dataset may include: determining a median trajectory length based on the trajectory length of each vehicle trajectory data in the second vehicle trajectory dataset, and determining the median trajectory length as the standard trajectory length. Alternatively, determining a mean trajectory length based on the trajectory length of each vehicle trajectory data in the second vehicle trajectory dataset, and determining the mean trajectory length as the standard trajectory length.

[0092] In some embodiments, clustering the third motor vehicle trajectory data set to obtain a cluster with a target number of centroids may include: randomly selecting a motor vehicle trajectory data from the third motor vehicle trajectory data set as the cluster centroid; determining the minimum distance between each motor vehicle trajectory data in the third motor vehicle trajectory data set except the cluster centroid and the cluster centroid; determining the probability that each motor vehicle trajectory data in the third motor vehicle trajectory data set except the cluster centroid is a new cluster centroid based on the minimum distance between each motor vehicle trajectory data in the third motor vehicle trajectory data set except the cluster centroid and the cluster centroid, and using the motor vehicle trajectory data corresponding to the maximum probability as the new cluster centroid; returning to the step of determining the minimum distance between each motor vehicle trajectory data in the third motor vehicle trajectory data set except the cluster centroid and the cluster centroid until the number of determined cluster centroids reaches the target number; for all cluster centroids in the third motor vehicle trajectory data set except all cluster centroids, For each motor vehicle trajectory data other than the first one, determine the distance between the motor vehicle trajectory data and each cluster centroid, assign the motor vehicle trajectory data to the cluster centroid corresponding to the minimum distance, and obtain the initial cluster cluster with the target number of centroids; for each initial cluster cluster, determine the sum of the distances between each motor vehicle trajectory data in the initial cluster and the other motor vehicle trajectory data in the initial cluster; use the motor vehicle trajectory data corresponding to the minimum distance sum as the new cluster centroid of the initial cluster; determine the distance between the new cluster centroid of the initial cluster and the previous cluster centroid, and when the distance between the new cluster centroid of the initial cluster and the previous cluster centroid is greater than a preset distance threshold, return to the step of determining the sum of the distances between each motor vehicle trajectory data in the initial cluster and the other motor vehicle trajectory data in the initial cluster, until the distance between the new cluster centroid of the initial cluster and the previous cluster centroid is less than or equal to the preset distance threshold, and obtain the cluster cluster with the target number of centroids. It should be noted that the minimum distance between the vehicle trajectory data and the cluster centroid can be represented by Euclidean distance, Dynamic Time Warping (DTW) value or cosine similarity.

[0093] In some embodiments, the driving area map construction method provided by the present invention further includes: performing normalization processing on each motor vehicle trajectory data in the second motor vehicle trajectory data set to obtain a third motor vehicle trajectory data set; clustering the third motor vehicle trajectory data set in sequence based on each number of centroids within a preset centroid number range, and determining the intra-cluster sum of squared errors (SSE) corresponding to each number of centroids based on the clustering results; generating an SSE change trend graph based on the SSE corresponding to each number of centroids, the SSE change trend graph being used to describe the change trend of SSE as the number of centroids increases; determining the SSE change rate corresponding to each number of centroids based on the SSE change trend graph, and determining the number of centroids with an SSE change rate less than the preset change rate as the candidate number of centroids; and determining the target number of centroids based on the multiple candidate numbers of centroids. The preset range of centroids and the preset change rate can be set by the user, and the present embodiment does not specifically limit this. For example, the preset centroid data set includes [1,30]. This embodiment can adaptively determine the target number of centroids based on the motor vehicle trajectory data set, thereby improving the accuracy of subsequent clustering of the motor vehicle trajectory data set.

[0094] In some embodiments, determining the target number of centroids based on multiple candidate number of centroids may include: sorting the multiple candidate number of centroids in descending order to obtain a centroid number sequence, determining the middle candidate number of centroids in the centroid number sequence as the target number of centroids, or determining the first candidate number of centroids in the centroid number sequence as the target number of centroids, or determining the second candidate number of centroids in the centroid number sequence as the target number of centroids.

[0095] Sub-step S1034: Filter the clusters of the target centroid quantity according to the lane information of the target road section to obtain a target motor vehicle trajectory dataset corresponding to each route of at least one route of the target road section.

[0096] This embodiment filters out motor vehicle trajectory data with trajectory duration greater than or equal to a target duration threshold from the traffic participant trajectory dataset, thereby eliminating irregular trajectory data of pedestrians and non-motor vehicles, as well as trajectory data of motor vehicles that have been stationary for a long time and have incomplete trajectories, to improve the accuracy of the motor vehicle trajectory dataset and reduce the amount of data during clustering, thereby improving clustering efficiency and accuracy. Moreover, based on the lane information of the target road section, cluster clusters corresponding to each route in at least one route of the target road section can be filtered out from the cluster clusters of the target number of centroids, so that the final target motor vehicle trajectory dataset is more consistent with the corresponding actual route of the target road section. In this way, based on the target motor vehicle trajectory dataset, a driving area map of the actual route can be accurately constructed.

[0097] In some embodiments, screening the target number of centroid clusters based on the lane information of the target road section may include: determining multiple standard driving routes for motor vehicles on the target road section based on the lane information of the target road section; determining the motor vehicle driving routes corresponding to each cluster based on the motor vehicle trajectory data in each cluster; screening a unique cluster of motor vehicle driving routes from the target number of centroid clusters as a candidate cluster; screening at least one target cluster from the multiple candidate clusters to obtain at least one target motor vehicle trajectory data set, wherein the motor vehicle driving route corresponding to the target cluster is the same as any standard driving route. This embodiment can accurately screen the target number of centroid clusters to obtain a matching cluster corresponding to each of the at least one route of the target road section.

[0098] For example, from Figure 2 The vehicle trajectory data with a trajectory duration greater than or equal to the target duration threshold is filtered out from the traffic participant trajectory data set of the target road section shown in the figure. Then, the filtered vehicle trajectory data set is clustered to obtain 20 clusters. Figure 2 The lane information of the target road section shown can be obtained from these 20 clusters as follows Figure 8 The routes corresponding to the six target clusters shown include a right-hand straight route along the main road, a left-hand straight route along the main road, a route turning right from the branch road to the main road, a route turning left from the main road to the branch road, a route turning left from the branch road to the main road, and a route turning right from the main road to the branch road. Therefore, a target motor vehicle trajectory dataset corresponding to each of the six routes is finally obtained.

[0099] For example, from Figure 3 The vehicle trajectory data sets of the traffic participants on the target road section shown in the figure are screened out, and the vehicle trajectory data sets with a trajectory duration greater than or equal to the target duration threshold are then clustered to obtain 8 clusters. Figure 3 The lane information of the target road section shown can be obtained from these 8 clusters as follows Figure 9 The routes corresponding to the two target clusters shown include a left-going straight route along the main road and a right-going straight route along the main road. Therefore, a target motor vehicle trajectory dataset corresponding to each of the two routes is finally obtained.

[0100] Step S104: constructing a driving area map corresponding to each route in at least one route of the target road section according to the target motor vehicle trajectory dataset corresponding to each route.

[0101] This embodiment uses a target motor vehicle trajectory dataset obtained from a target bird's-eye view image sequence corresponding to the traffic flow of the target road section to construct a driving area map for the target road section. This does not rely on the lane markings of the target road section, nor does it require motor vehicles to strictly follow the lane markings in the target road section. Therefore, regardless of whether the target road section is a structured section with clear lane markings and good traffic order, an unstructured section with blurred or even no lane markings, or an unstructured section where motor vehicles do not strictly follow lane markings, the target motor vehicle trajectory dataset obtained from the target bird's-eye view image sequence corresponding to the traffic flow of the target road section can be used to accurately construct a driving area map for the target road section, effectively improving the accuracy and universality of driving area map construction.

[0102] In some embodiments, constructing a driving area map corresponding to each route of at least one route of a target road segment based on a target vehicle trajectory dataset corresponding to each route may include: for each route, generating a vehicle trajectory image corresponding to the route based on the target vehicle trajectory dataset corresponding to the route, extracting a driving area contour for the route from the vehicle trajectory image; smoothing the driving area contour to obtain a target driving area contour, and filling the driving area contained in the target driving area contour to obtain a driving area map corresponding to the route. The driving area contour may be smoothed using a Gaussian filter. This embodiment, by converting the target vehicle trajectory dataset into a vehicle trajectory image before constructing the driving area map, does not rely on lane markings on the target road segment, nor does it require vehicles to strictly follow lane markings on the target road segment. This allows accurate construction of a driving area map for the corresponding route, regardless of whether the target road segment is a structured section with clear lane markings and good traffic order, an unstructured section with blurred or even no lane markings, or an unstructured section where vehicles do not strictly follow lane markings, effectively improving the accuracy and universality of driving area map construction.

[0103] In some embodiments, generating a vehicle trajectory image corresponding to a route based on a target vehicle trajectory dataset corresponding to the route may include: drawing each target vehicle trajectory data in the target vehicle trajectory dataset corresponding to the route in a blank image to obtain a vehicle trajectory image corresponding to the route.

[0104] In some embodiments, extracting a route's travel area outline from a vehicle trajectory image may include: binarizing the vehicle trajectory image to obtain a binary image; performing image erosion on the binary image to obtain a candidate binary image; detecting outliers in the candidate binary image using an outlier detection algorithm; and deleting outliers from the candidate binary image to obtain a target binary image; and performing image dilation on the target binary image to obtain the route's travel area outline. This embodiment, through image erosion and outlier processing, can effectively remove noise from the trajectory and improve the accuracy of the extracted travel area outline.

[0105] In some embodiments, the outlier detection algorithm may include a DBSCAN algorithm. Using the outlier detection algorithm, detecting outliers in a candidate binary image may include obtaining a neighborhood radius and a minimum number of points, and executing the DBSCAN algorithm on the candidate binary image based on the neighborhood radius and the minimum number of points to obtain outliers in the candidate binary image. The neighborhood radius and the minimum number of points may be user-configurable and are not specifically limited in the present embodiment. For example, the neighborhood radius may be 10 pixels and the minimum number of points may be 6.

[0106] In some embodiments, the size of the structuring element used in the image dilation process for the target binary image is determined based on the average width of the vehicles in the target vehicle trajectory dataset and the size of the pixels contracted due to image erosion. For example, the size of the structuring element used in the image dilation process for the target binary image is determined to be half the average width of the vehicles in the target vehicle trajectory dataset and twice the sum of the size of the pixels contracted due to image erosion.

[0107] In some embodiments, filling the driving area included in the target driving area outline may include: taking each white pixel point on the target driving area outline as a starting point, and for each starting point, taking the up, down, left, and right sides of the starting point as expansion directions; for each expansion direction, performing a traversal search between the starting point and the boundary of the target driving area outline along the expansion direction, and if another white pixel point different from the starting point is found, marking the expansion direction, recording the starting point, and recording the other white pixel point different from the starting point as the end point; if another white pixel point different from the starting point is not found, determining whether the boundary of the target driving area outline has been searched; if the boundary of the target driving area outline has been searched, determining whether the traversal of the four expansion directions has been completed; if the traversal of the four expansion directions has been completed, determining whether the number of marked expansion directions is greater than 1, and if the number of marked expansion directions is greater than 1, filling all pixel points from each starting point along the marked expansion direction to the corresponding end point to obtain a target driving area map.

[0108] The following generates Figure 8The process of constructing the driving area map is described by taking the driving area map corresponding to the route from the branch road to the main road as an example. Figure 8 Each target vehicle trajectory data in the target vehicle trajectory dataset corresponding to the route of turning left from the branch road to the main road is drawn in the blank image, and the Figure 8 The vehicle trajectory image corresponding to the route from the branch road to the main road is then Figure 8 The vehicle trajectory image corresponding to the route from the branch road to the main road is binarized and the Figure 10 a shows the binary image; Figure 10 The binary image shown in a is subjected to image corrosion processing to obtain Figure 10 b shows the candidate binary image; using the outlier detection algorithm, detect Figure 10 The outliers in the candidate binary image shown in b are Figure 10 Remove the outliers from the candidate binary image shown in b and get Figure 10 c shows the target binary image; Figure 10 The target binary image shown in c is subjected to image dilation processing to obtain Figure 10 d shows the driving area outline (the driving area outline of the route from the branch road to the main road); Figure 10 The contour of the driving area shown in d is smoothed to obtain Figure 10 e shows the target driving area outline; Figure 10 Fill the driving area contained in the target driving area outline shown in e, and get Figure 10 The driving area map shown in f (driving area map for turning left from a branch road to a main road).

[0109] In the same way, we can construct Figure 8 The remaining routes in the driving area map, such as Figure 11 As shown, Figure 11 The driving area map a in Figure 8 The right-hand straight route along the main road corresponds to: Figure 11 The driving area map b in Figure 8 The left straight route along the main road corresponds to: Figure 11 The driving area map c in Figure 8 The route of turning right from the branch road to the main road corresponds to: Figure 11 The driving area map d in Figure 8 The route of turning left from the main road to the branch road corresponds to: Figure 11 Driving area map in e and Figure 8 The route of turning left from the branch road to the main road corresponds to: Figure 11 The driving area map f in Figure 8In the same way, we can construct Figure 9 The driving area map of the route in Figure 12 As shown, Figure 12 The driving area map a in Figure 9 The left straight route along the main road corresponds to: Figure 12 The driving area map b in Figure 9 The route corresponds to the straight right route along the main road.

[0110] The driving area map constructed in the embodiment of the present invention is different from the standard lane area map. The driving area map is larger than the lane area map, which indirectly reflects the irregular driving behavior in the mixed traffic scene in the urban area. For specific differences, please refer to Figure 8 The area difference between the driving area map and the lane area map for the left-hand straight route along the main road, such as Figure 13 As shown, Figure 8 The boundary of driving area map 21 for the left-hand straight route along the main road is represented by a solid line, while the boundary of standard lane area map 22 is represented by a dashed line. Driving area map 21 essentially covers the area of ​​standard lane area map 22, and also covers part of the oncoming lane near the intersection. Preliminary analysis suggests that this phenomenon is caused in part by the left-hand lane being close to the turning area of ​​the intersection. To maintain a safe distance from turning vehicles on the branch road, most vehicles choose to use the oncoming lane to pass through the intersection. Furthermore, pedestrians frequently cross the street near the intersection, so vehicles also choose to pass through the relatively unobstructed oncoming lane. Compared to standard lane area map 22, driving area map 21 accurately reflects the driving area for the corresponding route in mixed pedestrian and pedestrian traffic scenarios in urban areas. When searching for traffic participants that influence the decision-making of motor vehicles on a given route, searching within the driving area can significantly reduce search time and computational costs.

[0111] See also Figure 14 , Figure 14 This is a schematic block diagram of the structure of a computer device provided by an embodiment of the present invention.

[0112] like Figure 14 As shown, the computer device 100 includes a processor 101 and a memory 102 , and the processor 101 and the memory 102 are connected via a bus 103 , such as an I 2 C (Inter-integrated Circuit) bus.

[0113] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire computer device. The processor 101 can be a central processing unit (CPU), and the processor 101 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0114] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0115] Those skilled in the art will understand that Figure 14 The structure shown in the figure is merely a block diagram of a portion of the structure related to the embodiment of the present invention, and does not constitute a limitation on the computer device to which the embodiment of the present invention is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0116] The processor 101 is configured to run a computer program stored in the memory 102 and implement any one of the methods for constructing a driving area map based on a bird's-eye view image provided by the embodiments of the present invention when executing the computer program.

[0117] In some embodiments, the processor 101 is configured to run a computer program stored in a memory, and implement the following steps when executing the computer program:

[0118] Obtain a target bird's-eye view image sequence corresponding to the traffic flow of the target road section;

[0119] Acquiring a traffic participant trajectory dataset of the target road section according to the target bird's-eye view image sequence;

[0120] Screening and classifying the traffic participant trajectory dataset to obtain a target motor vehicle trajectory dataset corresponding to each of the at least one route of the target road section;

[0121] A driving area map corresponding to each of the at least one route of the target road section is constructed based on the target motor vehicle trajectory dataset corresponding to each of the routes.

[0122] In some embodiments, when constructing a driving area map corresponding to each of the at least one route of the target road segment based on the target motor vehicle trajectory dataset corresponding to each of the routes, the processor 101 is configured to implement:

[0123] For each of the routes, generating a vehicle trajectory image corresponding to the route based on a target vehicle trajectory dataset corresponding to the route, and extracting a driving area contour of the route from the vehicle trajectory image;

[0124] The driving area outline is smoothed to obtain a target driving area outline, and the driving area included in the target driving area outline is filled to obtain a driving area map corresponding to the route.

[0125] In some embodiments, when extracting the driving area contour of the route from the motor vehicle trajectory image, the processor 101 is configured to implement:

[0126] performing a binarization process on the vehicle trajectory image to obtain a binarized image, and performing an image corrosion process on the binarized image to obtain a candidate binarized image;

[0127] Detecting outliers in the candidate binary image using an outlier detection algorithm, and deleting the outliers in the candidate binary image to obtain a target binary image;

[0128] Perform image expansion processing on the target binary image to obtain the driving area outline of the route.

[0129] In some embodiments, when the processor 101 filters and classifies the traffic participant trajectory dataset to obtain a target motor vehicle trajectory dataset corresponding to each of the at least one route of the target road segment, it is configured to:

[0130] Filtering motor vehicle trajectory data from the traffic participant trajectory dataset to obtain a first motor vehicle trajectory dataset;

[0131] Filtering out motor vehicle trajectory data having a trajectory duration greater than or equal to a target duration threshold from the first motor vehicle trajectory dataset to obtain a second motor vehicle trajectory dataset;

[0132] performing clustering processing on the second motor vehicle trajectory dataset to obtain clusters having the same number of target centroids, wherein the clusters are subsets of the second motor vehicle trajectory dataset;

[0133] According to the lane information of the target road section, the clusters of the target centroid number are screened to obtain a target motor vehicle trajectory data set corresponding to each of the at least one route of the target road section.

[0134] In some embodiments, when the processor 101 performs clustering processing on the second motor vehicle trajectory dataset to obtain clusters with the target number of centroids, it is configured to implement:

[0135] performing normalization processing on each motor vehicle trajectory data in the second motor vehicle trajectory dataset to obtain a third motor vehicle trajectory dataset;

[0136] Clustering is performed on the third motor vehicle trajectory dataset to obtain clusters with the same number of target centroids, where the clusters are subsets of the third motor vehicle trajectory dataset.

[0137] In some embodiments, when the processor 101 performs normalization processing on each vehicle trajectory data in the second vehicle trajectory dataset to obtain the third vehicle trajectory dataset, it is configured to implement:

[0138] unifying the trajectory lengths of each motor vehicle trajectory data in the second motor vehicle trajectory dataset to obtain a candidate motor vehicle trajectory dataset;

[0139] Perform mean-variance normalization processing on each motor vehicle trajectory data in the candidate motor vehicle trajectory dataset to obtain a third motor vehicle trajectory dataset.

[0140] In some embodiments, when the processor 101 implements screening of the clusters of the target number of centroids according to the lane information of the target road section, it is configured to implement:

[0141] determining a plurality of standard driving routes for a motor vehicle on the target road section according to lane information of the target road section;

[0142] Determining the motor vehicle driving routes corresponding to each cluster according to the motor vehicle trajectory data in each cluster;

[0143] Selecting the unique cluster of the motor vehicle driving route from the clusters of the target number of centroids as a candidate cluster;

[0144] At least one target cluster is selected from the plurality of candidate clusters to obtain at least one target motor vehicle trajectory dataset, wherein the motor vehicle driving route corresponding to the target cluster is the same as any of the standard driving routes.

[0145] In some embodiments, the processor 101 is further configured to implement:

[0146] performing normalization processing on each motor vehicle trajectory data in the second motor vehicle trajectory dataset to obtain a third motor vehicle trajectory dataset;

[0147] performing clustering processing on the third motor vehicle trajectory dataset based on each number of centroids within a preset centroid number range, and determining, according to the clustering result, a sum of squared errors (SSE) within the cluster corresponding to each number of centroids;

[0148] Generate an SSE change trend graph according to the SSE corresponding to each number of the centroids, wherein the SSE change trend graph is used to describe the change trend of the SSE as the number of the centroids increases;

[0149] Determine the SSE change rate corresponding to each of the number of centroids according to the SSE change trend graph, and determine the number of centroids whose SSE change rate is less than a preset change rate as the candidate number of centroids;

[0150] The target number of centroids is determined based on the number of candidate centroids.

[0151] In some embodiments, when acquiring the traffic participant trajectory dataset of the target road section according to the target bird's-eye view image sequence, the processor 101 is configured to implement:

[0152] calling a preset traffic participant detection model to perform traffic participant detection on each bird's-eye view image in the target bird's-eye view image sequence, and obtaining a traffic participant detection result for each bird's-eye view image;

[0153] determining static data of each traffic participant in each of the bird's-eye view images according to the traffic participant detection result of each of the bird's-eye view images;

[0154] performing multi-target tracking on the traffic participants in each of the bird's-eye view images based on static data of each traffic participant in each of the bird's-eye view images, and assigning a tracking ID to each of the traffic participants that is successfully tracked;

[0155] According to the tracking ID of each traffic participant, the static data of each traffic participant in each bird's-eye view image are associated to obtain a static data sequence corresponding to each tracking ID;

[0156] The static data in the static data sequence corresponding to each tracking ID is corrected to obtain a target static data sequence corresponding to each tracking ID, and each target static data sequence is aggregated to obtain a traffic participant trajectory dataset of the target road section.

[0157] In some embodiments, when correcting the static data in the static data sequence corresponding to the tracking ID to obtain the target static data sequence corresponding to the tracking ID, the processor 101 is configured to implement:

[0158] Obtaining a directional detection frame closest to a minimum perspective distortion region from the directional detection frames included in each static data in the static data sequence as a reference detection frame, and determining a size of the reference detection frame as a correction reference size;

[0159] Determining, among the directional detection frames included in each static data in the static data sequence, excluding the reference detection frame, as detection frames to be corrected;

[0160] For each of the detection frames to be corrected, determining a corner point closest to the reference detection frame among the four corner points of the detection frame to be corrected as a correction reference corner point;

[0161] According to the correction reference corner points and the correction reference size of each detection frame to be corrected, the position and shape of each detection frame to be corrected in the static data sequence are corrected to obtain the target static data sequence.

[0162] In some embodiments, the traffic participant detection model is obtained by iteratively training a target detection model in advance based on a training sample data set, where the sample data in the training sample set includes sample images and annotated oriented detection boxes, orientation angles, and traffic participant categories.

[0163] In some embodiments, when acquiring a target bird's-eye view image sequence corresponding to a traffic flow of a target road section, the processor 101 is configured to implement:

[0164] Acquire a plurality of original bird's-eye view image sequences obtained by an image acquisition device capturing the traffic flow of the target road section from a bird's-eye view perspective;

[0165] Performing image stabilization processing on each of the multiple original bird's-eye view image sequences, and aligning the multiple original bird's-eye view image sequences after image stabilization processing to the same image coordinate system to obtain the multiple target bird's-eye view image sequences.

[0166] It should be noted that, those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working process of the computer device described above can refer to the corresponding process in the aforementioned embodiment of the driving area map construction method based on bird's-eye view images, and will not be repeated here.

[0167] See also Figure 15 , Figure 15This is a schematic block diagram of the structure of a driving area map construction system based on a bird's-eye view image provided by an embodiment of the present invention.

[0168] like Figure 15 As shown, the system 1000 for constructing a driving area map based on a bird's-eye view image includes a computer device 100, a height maintenance device 200, and an image acquisition device 300, wherein:

[0169] The height maintaining device 200 is deployed on the target road section and is used to carry the image acquisition device 300, so that the image acquisition device 300 can acquire a plurality of original bird's-eye view image sequences of the traffic flow of the target road section from a bird's-eye view perspective;

[0170] The image acquisition device 300 is mounted on the altitude maintaining device 200 and is used to acquire a plurality of original bird's-eye view image sequences of the traffic flow of the target road section from a bird's-eye view perspective;

[0171] The computer device 100 is configured to obtain the plurality of original bird's-eye view image sequences and implement any one of the methods for constructing a driving area map based on bird's-eye view images provided in the embodiments of the present invention.

[0172] In some embodiments, the altitude maintaining device 200 may include an unmanned aerial vehicle, a stand, or a street lamp, and the image acquisition device 300 may include a digital camera, a single-lens reflex camera, an infrared camera, or a depth camera.

[0173] In some embodiments, the system 1000 for constructing a driving area map based on bird's-eye images further includes an image storage device for storing a plurality of original bird's-eye image sequences obtained by the image acquisition device 300 from capturing the traffic flow of the target road section from a bird's-eye perspective. The unmanned aerial vehicle is communicatively connected to the image storage device so that the unmanned aerial vehicle transmits the plurality of original bird's-eye image sequences obtained by the image acquisition device 300 from capturing the traffic flow of the target road section from a bird's-eye perspective to the image storage device for storage. Alternatively, the image acquisition device is communicatively connected to the image storage device so that the image acquisition device transmits the plurality of original bird's-eye image sequences obtained by capturing the traffic flow of the target road section from a bird's-eye perspective to the image storage device for storage.

[0174] It should be noted that, those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the driving area map construction system based on the bird's-eye view image described above can refer to the corresponding process in the aforementioned driving area map construction method embodiment based on the bird's-eye view image, and will not be repeated here.

[0175] An embodiment of the present invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any of the methods for constructing a driving area map based on a bird's-eye view image as provided in the description of the embodiment of the present invention.

[0176] The storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device.

[0177] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0178] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0179] The serial numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope of protection of the claims.

Claims

1. A method for constructing a driving area map based on a bird's-eye view image, characterized in that: include: Obtain a target bird's-eye view image sequence corresponding to the traffic flow of the target road section; Acquiring a traffic participant trajectory dataset of the target road section according to the target bird's-eye view image sequence; Screening and classifying the traffic participant trajectory dataset to obtain a target motor vehicle trajectory dataset corresponding to each of the at least one route of the target road section; A driving area map corresponding to each of the at least one route of the target road section is constructed based on the target motor vehicle trajectory dataset corresponding to each of the routes.

2. The driving area map construction method according to claim 1, characterized in that: The step of constructing a driving area map corresponding to each of the at least one route of the target road section based on the target motor vehicle trajectory dataset corresponding to each of the routes comprises: For each of the routes, generating a vehicle trajectory image corresponding to the route based on a target vehicle trajectory dataset corresponding to the route, and extracting a driving area contour of the route from the vehicle trajectory image; The driving area outline is smoothed to obtain a target driving area outline, and the driving area included in the target driving area outline is filled to obtain a driving area map corresponding to the route.

3. The method for constructing a driving area map according to claim 2, wherein: The step of extracting the travel area contour of the route from the motor vehicle trajectory image comprises: performing a binarization process on the vehicle trajectory image to obtain a binarized image, and performing an image corrosion process on the binarized image to obtain a candidate binarized image; Detecting outliers in the candidate binary image using an outlier detection algorithm, and deleting the outliers in the candidate binary image to obtain a target binary image; Perform image expansion processing on the target binary image to obtain the driving area outline of the route.

4. The method for constructing a driving area map according to claim 1, wherein: The screening and classification of the traffic participant trajectory dataset to obtain a target motor vehicle trajectory dataset corresponding to each of the at least one route of the target road section includes: Filtering motor vehicle trajectory data from the traffic participant trajectory dataset to obtain a first motor vehicle trajectory dataset; Filtering out motor vehicle trajectory data having a trajectory duration greater than or equal to a target duration threshold from the first motor vehicle trajectory dataset to obtain a second motor vehicle trajectory dataset; performing clustering processing on the second motor vehicle trajectory dataset to obtain clusters having the same number of target centroids, wherein the clusters are subsets of the second motor vehicle trajectory dataset; According to the lane information of the target road section, the clusters of the target centroid number are screened to obtain a target motor vehicle trajectory data set corresponding to each of the at least one route of the target road section.

5. The method for constructing a driving area map according to claim 4, characterized in that: The clustering process of the second motor vehicle trajectory dataset to obtain the number of clusters of target centroids includes: performing normalization processing on each motor vehicle trajectory data in the second motor vehicle trajectory dataset to obtain a third motor vehicle trajectory dataset; Clustering is performed on the third motor vehicle trajectory dataset to obtain clusters with the same number of target centroids, where the clusters are subsets of the third motor vehicle trajectory dataset.

6. The method for constructing a driving area map according to claim 5, characterized in that: The step of normalizing each vehicle trajectory data in the second vehicle trajectory dataset to obtain a third vehicle trajectory dataset includes: unifying the trajectory lengths of each motor vehicle trajectory data in the second motor vehicle trajectory dataset to obtain a candidate motor vehicle trajectory dataset; Perform mean-variance normalization processing on each motor vehicle trajectory data in the candidate motor vehicle trajectory dataset to obtain a third motor vehicle trajectory dataset.

7. The method for constructing a driving area map according to claim 4, wherein: The screening of the clusters of the target number of centroids according to the lane information of the target road section includes: determining a plurality of standard driving routes for a motor vehicle on the target road section according to lane information of the target road section; Determining the motor vehicle driving routes corresponding to each cluster according to the motor vehicle trajectory data in each cluster; Selecting the unique cluster of the motor vehicle driving route from the clusters of the target number of centroids as a candidate cluster; At least one target cluster is selected from the plurality of candidate clusters to obtain at least one target motor vehicle trajectory dataset, wherein the motor vehicle driving route corresponding to the target cluster is the same as any of the standard driving routes.

8. The method for constructing a driving area map according to claim 4, wherein: The method further comprises: performing normalization processing on each motor vehicle trajectory data in the second motor vehicle trajectory dataset to obtain a third motor vehicle trajectory dataset; performing clustering processing on the third motor vehicle trajectory dataset based on each number of centroids within a preset centroid number range, and determining, according to the clustering result, a sum of squared errors (SSE) within the cluster corresponding to each number of centroids; Generate an SSE change trend graph according to the SSE corresponding to each number of the centroids, wherein the SSE change trend graph is used to describe the change trend of the SSE as the number of the centroids increases; Determine the SSE change rate corresponding to each of the number of centroids according to the SSE change trend graph, and determine the number of centroids whose SSE change rate is less than a preset change rate as the candidate number of centroids; The target number of centroids is determined based on the number of candidate centroids.

9. The method for constructing a driving area map according to any one of claims 1 to 8, characterized in that: The step of obtaining a traffic participant trajectory dataset of the target road section according to the target bird's-eye view image sequence includes: calling a preset traffic participant detection model to perform traffic participant detection on each bird's-eye view image in the target bird's-eye view image sequence, and obtaining a traffic participant detection result for each bird's-eye view image; determining static data of each traffic participant in each of the bird's-eye view images according to the traffic participant detection result of each of the bird's-eye view images; performing multi-target tracking on the traffic participants in each of the bird's-eye view images based on static data of each traffic participant in each of the bird's-eye view images, and assigning a tracking ID to each of the traffic participants that is successfully tracked; According to the tracking ID of each traffic participant, the static data of each traffic participant in each bird's-eye view image are associated to obtain a static data sequence corresponding to each tracking ID; The static data in the static data sequence corresponding to each tracking ID is corrected to obtain a target static data sequence corresponding to each tracking ID, and each target static data sequence is aggregated to obtain a traffic participant trajectory dataset of the target road section.

10. The method for constructing a driving area map according to claim 9, wherein: Correcting the static data in the static data sequence corresponding to the tracking ID to obtain a target static data sequence corresponding to the tracking ID includes: Obtaining a directional detection frame closest to a minimum perspective distortion region from the directional detection frames included in each static data in the static data sequence as a reference detection frame, and determining a size of the reference detection frame as a correction reference size; Determining, among the directional detection frames included in each static data in the static data sequence, excluding the reference detection frame, as detection frames to be corrected; For each of the detection frames to be corrected, determining a corner point closest to the reference detection frame among the four corner points of the detection frame to be corrected as a correction reference corner point; According to the correction reference corner points and the correction reference size of each detection frame to be corrected, the position and shape of each detection frame to be corrected in the static data sequence are corrected to obtain the target static data sequence.

11. The method for constructing a driving area map according to claim 9, wherein: The traffic participant detection model is obtained by iteratively training a target detection model in advance based on a training sample data set, where the sample data in the training sample set includes sample images and annotated oriented detection boxes, orientation angles, and traffic participant categories.

12. The method for constructing a driving area map according to any one of claims 1 to 8, characterized in that: The step of obtaining a target bird's-eye view image sequence corresponding to the traffic flow of the target road section includes: Acquire a plurality of original bird's-eye view image sequences obtained by an image acquisition device capturing the traffic flow of the target road section from a bird's-eye view perspective; Performing image stabilization processing on each of the multiple original bird's-eye view image sequences, and aligning the multiple original bird's-eye view image sequences after image stabilization processing to the same image coordinate system to obtain the multiple target bird's-eye view image sequences.

13. A computer device, characterized in that: The computer device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the driving area map construction method according to any one of claims 1 to 12 is implemented.

14. A driving area map construction system based on bird's-eye view images, characterized in that: include: Altitude maintenance equipment, image acquisition equipment and computer equipment, including: The height maintaining device is deployed on the target road section and is used to carry the image acquisition device, so that the image acquisition device can acquire a plurality of original bird's-eye view image sequences of the traffic flow of the target road section from a bird's-eye view perspective; The image acquisition device is mounted on the altitude maintaining device and is used to acquire a plurality of original bird's-eye view image sequences of the traffic flow of the target road section from a bird's-eye view perspective; The computer device is used to obtain the multiple original bird's-eye view image sequences and implement the driving area map construction method according to any one of claims 1 to 12.

15. A storage medium for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the driving area map construction method according to any one of claims 1 to 12.

Images